Reviewing Grant Proposals

I recently served on a panel reading NSF grant proposals.  We were admonished not only to respect the confidentiality of the proposal contents but also not to divulge the dates we did the reviews nor the title or nature of the grant types being solicited.

I've reviewed NSF grant proposals on about a half dozen occasions, and I've enjoyed each of my experiences.   There are typically 4 or 5 members to a committee, a few dozen committees representing the different STEM disciplines, and each committee is assigned about a dozen proposals to review.  NSF tries to create panels with a diversity of geographical regions and of institution type  (research universities, state universities, liberal arts colleges, and sometimes even two-year colleges). NSF brings in both faculty and administrators.

Panelists register online using the NSF program FastLane long before they see any of the proposals. FastLane facilitates many aspects of the review process, such as tracking panelist coordinates and other information for  travel arrangements, giving access to proposals, recording panelist reviews, and arranging letters to home institutions acknowledging service.

Panelists upload to Fastlane their personal evaluations of each proposal before going to DC and physically meeting as a panel.  Project descriptions are restricted to 15 pages, but with all the additional documentation of references, biographical sketches, budget, etc., the entire proposal usually exceeds 50 pages.  So it is also common that when the panels finally meet in DC on the first morning, not all the panelists have successfully prepared reviews of all the proposals.

The panels meet essentially all of the first day discussing each proposal.  Panelists are logged on the FastLane program during discussion.  This allows each panelist the ability to see the reviews of all the panelists for each proposal.

NSF assigns each panelist to be "scribe" for two or three proposals, and the scribe's task is to record the discussion about the proposal.  NSF does not require consensus, but it is common that the discussions persuade some panelists to change their initial evaluations.  Most panelists work into the night preparing the summaries required by their duties as scribe and resubmitting their personal proposal reviews.

The second morning is spent largely in approving the scribe summaries.  Again the panelists are logged on FastLane, and they are not allowed to leave until each summary has been approved.  NSF reconvenes the panels by discipline late in the morning for a debriefing before sending the panelists back home.

The debriefing is one of the highlights of the whole experience, because each panel typically describes its two favorite proposals.  So we hear the highlights, and it's exciting to learn of innovative ideas that are being pursued.