Results of the NIH funding survey
by: Denise Kirschner and Ramit Mehr
There seems to be a general feeling among researchers in mathematical biology,
whose research is specifically in health-related areas, that it is inordinately difficult to obtain funding from the NIH. This seems to be based primarily on the nature of the review process of proposals in theoretical or mathematical biology. If this is the case, and if we can document statistics, then we may be able to rectify the situation by, first, showing the NIH that we do submit a significant number of proposals (and promise to in the future) and, second,
that we do not receive a proper review to be funded. Hence, we could request a study section to be established to review mathematical biology proposals.
As a first step, last fall we have published a survey asking SMB members about
their experiences with NIH funding. We now present the results of this survey,
which will lead to the next step, that is, a discussion and decision about the
action required. The quantifiable results are presented in Table 1. It is clear that there is indeed a significant number of mathematical biology proposals.
It is also clear that there would have been many more proposals if researchers felt that these would get a fair, informed, helpful review, but, strikingly, most researchers felt that the reviews were NOT helpful.
Table: Compilation of members' responses to our NIH survey.
*The total number of replies were 50, out of which 20 (40%) were from the area of computational neuroscience. Hence we tabulate the results both
with and without the computational neuroscience replies.
1All those who haven't submitted grant proposal to NIH stated
that they didn't apply because they had heard it was impossible to get
funding from NIH due to the theoretical nature of the work.
2In reply to question 4a: The most popular funding section is
by NIMH and NINDS who set up a special study section for Computational
Neuroscience. In fact which is now well established in NIMH. Dennis
Glanzman is the program director. Other study sections were: NIGMS
(director: Marion Zatz); Allergy and Infectious Disease, Immunology,
Joseph Albright AIDS program; NHLI, NIDDK, DIV. RESEARCH RESOURCES;
NIDDK; Three Renal Program NIAMDK, Two Division Research Resources.
3Question 4b was: If a proposal(s) was rejected, did your
pink sheets reflect accurately the proposals quality? (Be honest!)
Specifically, was it able to address the mathematical modeling
presented? It is evident that almost all of the reviews were not
|1) Have you ever submitted a proposal to the NIH?1
||YES: 43, NO: 7
||YES: 23, NO: 7
|2) Average # of proposals submitted over the past 10 years:
|3) Average # of proposals
funded2over the past 10 years:
|4b) Review Quality:3
|Low funding range
|No pink sheets:
|Not helpful review:
Survey participants were also asked to share any other thoughts on NIH funding they may have, based on their experiences. Comments usually fell under one of the following three categories:
(a) Responses from people who were on NIH study teams, explaining how these
work in practice.
(b) Responses that consist of feedback to NIH (criticism and suggestions).
(c) Responses from the above or people who succeeded in obtaining funding,
containing advice to future applicants.
In the following, we give a compilation of the main messages within each category.
How NIH study teams work
- ``NIH seems willing to set up special review panels for theoretically
oriented proposals. I have served on a few. They were representative of
the field, well run, and fair. However, I do not know what happens after
the special review panel, when the reviewed proposal goes into
competition with proposals that went through the regular study sections."
- ``There is a branch of NIH called NCRR that is supposed to focus on
mathematical modeling among other things. My proposal on cardiac modeling
ended up at NHBLI which is a good place considering the scientific
content of the proposal. I am not sure what sorts of modeling proposals
go to NCRR."
- ``Overall the real problem is that the percentile required for funding has
varied between 13 and 25. This means that a large fraction of grants
require a second and sometimes third go around. This is very disruptive
on the practice of science because labs fall apart, promising post docs
take jobs in industry. Almost everyone I know has had problems with
funding. I am not at all sure modelers have been singled out."
- ``No modeling proposal succeeds without a powerful advocate on the study
- ``There is an issue with the split between intramural and extramural
funding of biocomputing. In general, the Institutes spend about 15
their budget intramurally and 85
If you look at biocomputing broadly, the intramural components (DCRT,
NLM, NCBI, NCIFCRF) consume a far larger slice of the pie than is the
Feedback from applicants to NIH
``My impression is that the NIH funding is not geared for biomathematics.
It does not exclude biomaths but it is not very forthcoming. Most
importantly, I have the impression that the reviewing process is
somewhat biased towards powerful, but not necessarily better scientists."
``Usually there are individuals on the study sections who understand the
math. Frequently these are ad hoc reviewers without much clout with
regular members of a study section."
``My experience with a recent ad hoc committee is that all reviewers try
their best to do a thorough job, but that there are serious
inconsistencies in the relative scores assigned by reviewers with
different backgrounds (e.g., mathematicians vs. data people), even when
the written comments are very similar. Furthermore, the people who had
the least background relevant to the proposal gave significantly lower
scores than the people with more direct expertise. Because funding
ultimately depends on the numeric priority score, this difference in
relative rankings needs to be addressed. For the proposal that was
rejected, I never revised. I put a lot of work into it and got the
distinct impression from my initial review that there would be little
interest on the part of the study section in funding the proposal. (There
were originally two modelers on the study section. One resigned shortly
before I submitted my proposal, the other had a competing proposal in the
same round and could not review mine. So I was left at the mercy of the
``The other comment one hears often from NIH proposers is that the system
is extremely conservative - the pink sheets often instruct the
investigator to leave off everything that is speculative, and do only the
next routine step (usually boring to the investigator). Its a very
different mindset from NSF.
This could be especially hard on mathematical proposals, which cannot
spell out every step to be done - if one could, it wouldn't be worth
doing - unlike a straight experimental proposal. NIH is aware of this one
too, but what it will do about this ....? "
- Arguments for a separate study section for mathematical biology:
``I believe a standing study section to review modeling proposals is
essential and have argued this for 20 years. I have been a member of
study sections and have had to explain the mathematical aspects to other
members and explain why it was important to do correct quantitative
``I doubt I would have had a chance at funding if NIMH had not been wise
enough to set up the Computational Neuroscience program."
``In every case the proposal succeeded or failed based on the experimental
content. Even when a single modeling expert is recruited as an ad hoc
reviewer, his or her opinions are overridden by the majority of study
section members who know little and care less about modeling. I have
seen this both as a member of the review group and as an applicant."
- Arguments against a separate study section for mathematical biology (and
for inclusion of modelers in specific study sections):
``The problem with a modeling study section is that the underlying biology
may be equally misunderstood. It might be more to the point to try to
get some modelers with expertise in a given area on regular study
sections. However interdisciplinary funding has always been given more
lip service than funding. "
``Since math biology is applied to all areas of biology, I don't believe
there could be a math biology study section. The more important thing for a
study section to be able to do is to evaluate how important the proposed
biological problem is. I doubt whether a Math biology Study section could do
this with such a wide range of proposals submitted."
- Other suggestions:
``The reviews from biostatisticians were very unuseful - I strongly
believe that biostatisticians should not review modelling grants."
``Put NIH quantitative types (not statisticians) on every study section."
``Experimentalists evaluating theoretical/mathematical modeling proposals use
standards for experimental proposals. Could we get together and write a set of
theoretical guidelines? I have some I've used in the past. I think
experimentalists would largely be receptive."
``One possibility is to make up a data base of experts in theory that
specialize in various bio-fields - and then make this data base available to
the various study section. Perhaps this could be done under the auspices of
the SMB but also including Soc Biomed Engineers and Biophys Soc ... "
Advice to applicants
- ``Target the mission of NIH.
Identify a problem of interest to the biological/medical community.
Place the proposed work in an appropriate context, and
address the anticipated impact of the work on biomedical research.
Mathematics is just a methodology, just like micropuncture,
histochemistry, etc, which is brought to bear on a biological problem.
The study groups have little enthusiasm for mathematical novelty, but
rather focus on the payoff of the application."
- ``The narrower the proposal the better."
- ``Present enough background to show that you are
familiar with the currently literature and outstanding biological issues."
- ``Present significant amount of work in the form of preliminary studies
that clearly show the types of results that one can get from a model,
and that these results are interesting."
- ``The results should be testable, otherwise it amounts to speculation
rather than science."
- ``It is important to have experimental colleagues willing to discuss ideas
and test results of the models and to have them write letters of
- ``Unless a proposal is fronted by an
MD or has a high experimental component, I do not expect it will have
any chance for funding with the current NIH review structure."
- ``The proposal has to be written so that the experimental biologists can
understand it even if he or she cannot understand the math. Probably the
best choice of getting funded is to get a special study section. In the
cover letter one should emphasize tat for peer review, people with math
expertise must be on the study section."
- ``When revising a proposal: Persist and reapply to EVERY nuance of the
critique. Don't fight, just argue carefully, with evidence or re-write to do
what they wish."