In a recent article in ScienceInsider, Jeffrey Mervis reports that the National Science Foundation’s Commission on Merit Review will likely recommend that one of the key criteria used to rate a proposal–broader impacts—be renamed “societal benefits.” Currently, NSF proposals are rated on their “intellectual merit”, or their ability to advance knowledge, and their “broader impacts”, defined by NSF as “the potential to benefit society and contribute to the achievement of specific, desired societal outcomes.” While reviewers generally have little problem evaluating the intellectual merit of a proposal, the same is not always true of its broader impacts. Moreover, Mervis reports that there is a perception that “many reviewers give short shrift to the broader impacts criteria. . .” In addition to applying the label “societal benefits”, the merit commission is considering a change in which reviewers would give this criterion a separate score that would then be folded into a proposal’s overall rating.
The new emphasis on societal benefits fits well within CfAS’ mission and vision. Our goal is to use the ever-expanding quantity and quality of data on past societies to assist our society in solving the challenges facing us today. We look forward to the final version of the commission’s recommendations, and to working with our partners and associates to craft research that truly has “societal benefits.”
1 thought on “NSF proposing greater consideration be given to “Societal Benefits” in grant reviews”
I fully support the use of research to help solve modern issues. In fact, why else do research?
However, I have reviewed NSF proposals for many years, and 20+ years ago served on the panel. And for anyone reading this who has not written an NSF proposal, let me just say that it is the hardest thing that any researcher will do in their career.
Currently, NSF asks reviewers to consider a proposal’s impacts in terms of their potential to “benefit society or advance desired societal outcomes.” (Let’s leave aside whether we could all agree on what constitutes “desired” societal outcomes.) This is a tall order for NSF proposals, given that to be fundable a proposal must first pass the Intellectual Merit requirement and, in a world where a considerable amount of knowledge already exists on any given subject, a proposal has to cover a lot of bases to show how the proposed research is going to advance the field. (And NSF is very conservative: the advice I was given my first year of teaching by the university research office was “first, do half the research”. The reason for this advice was provided by a frustrated cancer researcher in a NYT editorial I read years later: NSF doesn’t give awards for research, it gives prizes. He couldn’t get funded through NSF, but he was ultimately successful because, with funding from Clairol, he created a major breast cancer drug.)
Because of the difficulty in passing muster under the Intellectual Merit category, the Broader Impacts often are tacked-on. They often claim that the research to be done will produce data sets that someone, somewhere, at sometime might find useful for something else; and, also, that students will be trained. That’s it. I’m not blaming the PIs: I’ve used these claims myself. But this is not what NSF envisions for broader impacts, and now it wishes to expand their importance (which, incidentally, NSF has said in the past; but I’ve seen no impact on the proposals I read, including one last week). They are, I heard years ago at an NSF workshop, something that should be woven through a proposal, and reflect some additional activity that, indeed, will (not might) result in a broader societal impact.
However, devising such broader impacts takes time, and requires funding. In proposals that have given thought to such impacts, and have woven them into the proposal, I find that the intellectual merit suffers – because time and money is being spent on something other than the research. And quality research (and that is all that NSF funds and should fund) is increasingly difficult to accomplish (e.g., North American researchers now have to deal with new NAGPRA requirements, which no lawyer really knows how to implement, that impact collections research). Doing a good job on the broader impacts (remember: NSF proposals are limited to 15 pages!) makes a proposal neither fish nor fowl, and a proposal’s broader impacts won’t matter if the research, the intellectual merit, suffers (because the proposal won’t be funded).
Broader impacts are important, but NSF should find another way to achieve them than by asking researchers to do two very different things at the same time in an environment of limited funding (and NSF’s funding for Archaeology is quite limited).
Maybe we need a separate competition, one that focuses just on “societal benefits” and makes use of what we’ve learned from research. This would almost certainly entail serious cross-disciplinary work. Imagine this: some have proposed (I think EO Wilson was one) taking some large chunk of the plains, e.g., portions of the Dakotas and Wyoming, where the human population is very low density today, and generally impoverished, and turn it back into open range for bison, etc. Maybe turn it over to a consortium of tribes to manage as a for-profit herd? (Also: tourism). What would it take to achieve this? I don’t know but I do know it would require a bison ecologist, a general plains ecologist, tribal officials, a climatologist perhaps, and archaeologists (OK, I’m biased) and, oh yeah, a political scientist and economist to figure out how to depopulate portions of states without destroying the state’s economy; or maybe we just fence off unpopulated federal land? I don’t know the answer, but I do know you could not possible embed such a venture into a 15 page archaeology proposal to research a new bison kill site.
But is this what NSF should be doing, or should this be the purview of some other agency, or a new directorate in NSF? And, if Republicans piss and moan about worthless research (and they have since the days of Senator Proxmire; and archaeology is often high on their list), what would they say about a directorate whose purpose is to achieve desired societal outcomes? I see no easy way forward.
My two cents.