Results of the NIH funding survey
by: Denise Kirschner and Ramit Mehr

There seems to be a general feeling among researchers in mathematical biology, whose research is specifically in health-related areas, that it is inordinately difficult to obtain funding from the NIH. This seems to be based primarily on the nature of the review process of proposals in theoretical or mathematical biology. If this is the case, and if we can document statistics, then we may be able to rectify the situation by, first, showing the NIH that we do submit a significant number of proposals (and promise to in the future) and, second, that we do not receive a proper review to be funded. Hence, we could request a study section to be established to review mathematical biology proposals.

As a first step, last fall we have published a survey asking SMB members about their experiences with NIH funding. We now present the results of this survey, which will lead to the next step, that is, a discussion and decision about the action required. The quantifiable results are presented in Table 1. It is clear that there is indeed a significant number of mathematical biology proposals. It is also clear that there would have been many more proposals if researchers felt that these would get a fair, informed, helpful review, but, strikingly, most researchers felt that the reviews were NOT helpful.

Table: Compilation of members' responses to our NIH survey. *The total number of replies were 50, out of which 20 (40%) were from the area of computational neuroscience. Hence we tabulate the results both with and without the computational neuroscience replies. 1All those who haven't submitted grant proposal to NIH stated that they didn't apply because they had heard it was impossible to get funding from NIH due to the theoretical nature of the work. 2In reply to question 4a: The most popular funding section is by NIMH and NINDS who set up a special study section for Computational Neuroscience. In fact which is now well established in NIMH. Dennis Glanzman is the program director. Other study sections were: NIGMS (director: Marion Zatz); Allergy and Infectious Disease, Immunology, Joseph Albright AIDS program; NHLI, NIDDK, DIV. RESEARCH RESOURCES; NIDDK; Three Renal Program NIAMDK, Two Division Research Resources. 3Question 4b was: If a proposal(s) was rejected, did your pink sheets reflect accurately the proposals quality? (Be honest!) Specifically, was it able to address the mathematical modeling presented? It is evident that almost all of the reviews were not helpful.
Question with NS without NS*
1) Have you ever submitted a proposal to the NIH?1 YES: 43, NO: 7 YES: 23, NO: 7
2) Average # of proposals submitted over the past 10 years: 5.1 plus or minus 6.1 4.8plus or
3) Average # of proposals funded2over the past 10 years: 2.8plus or
minus5.4 2.4plus or
4b) Review Quality:3    
Low funding range 1 1
No pink sheets: 3 3
Helpful review: 5 1
Not helpful review: 20 18

Survey participants were also asked to share any other thoughts on NIH funding they may have, based on their experiences. Comments usually fell under one of the following three categories: (a) Responses from people who were on NIH study teams, explaining how these work in practice. (b) Responses that consist of feedback to NIH (criticism and suggestions). (c) Responses from the above or people who succeeded in obtaining funding, containing advice to future applicants. In the following, we give a compilation of the main messages within each category.

How NIH study teams work
  • ``NIH seems willing to set up special review panels for theoretically oriented proposals. I have served on a few. They were representative of the field, well run, and fair. However, I do not know what happens after the special review panel, when the reviewed proposal goes into competition with proposals that went through the regular study sections."
  • ``There is a branch of NIH called NCRR that is supposed to focus on mathematical modeling among other things. My proposal on cardiac modeling ended up at NHBLI which is a good place considering the scientific content of the proposal. I am not sure what sorts of modeling proposals go to NCRR."
  • ``Overall the real problem is that the percentile required for funding has varied between 13 and 25. This means that a large fraction of grants require a second and sometimes third go around. This is very disruptive on the practice of science because labs fall apart, promising post docs take jobs in industry. Almost everyone I know has had problems with funding. I am not at all sure modelers have been singled out."
  • ``No modeling proposal succeeds without a powerful advocate on the study section."
  • ``There is an issue with the split between intramural and extramural funding of biocomputing. In general, the Institutes spend about 15 their budget intramurally and 85 If you look at biocomputing broadly, the intramural components (DCRT, NLM, NCBI, NCIFCRF) consume a far larger slice of the pie than is the norm."

Feedback from applicants to NIH
  • Criticism:

    ``My impression is that the NIH funding is not geared for biomathematics. It does not exclude biomaths but it is not very forthcoming. Most importantly, I have the impression that the reviewing process is somewhat biased towards powerful, but not necessarily better scientists."

    ``Usually there are individuals on the study sections who understand the math. Frequently these are ad hoc reviewers without much clout with regular members of a study section."

    ``My experience with a recent ad hoc committee is that all reviewers try their best to do a thorough job, but that there are serious inconsistencies in the relative scores assigned by reviewers with different backgrounds (e.g., mathematicians vs. data people), even when the written comments are very similar. Furthermore, the people who had the least background relevant to the proposal gave significantly lower scores than the people with more direct expertise. Because funding ultimately depends on the numeric priority score, this difference in relative rankings needs to be addressed. For the proposal that was rejected, I never revised. I put a lot of work into it and got the distinct impression from my initial review that there would be little interest on the part of the study section in funding the proposal. (There were originally two modelers on the study section. One resigned shortly before I submitted my proposal, the other had a competing proposal in the same round and could not review mine. So I was left at the mercy of the nonmodelers.)"

    ``The other comment one hears often from NIH proposers is that the system is extremely conservative - the pink sheets often instruct the investigator to leave off everything that is speculative, and do only the next routine step (usually boring to the investigator). Its a very different mindset from NSF. This could be especially hard on mathematical proposals, which cannot spell out every step to be done - if one could, it wouldn't be worth doing - unlike a straight experimental proposal. NIH is aware of this one too, but what it will do about this ....? "

  • Arguments for a separate study section for mathematical biology:

    ``I believe a standing study section to review modeling proposals is essential and have argued this for 20 years. I have been a member of study sections and have had to explain the mathematical aspects to other members and explain why it was important to do correct quantitative analysis. "

    ``I doubt I would have had a chance at funding if NIMH had not been wise enough to set up the Computational Neuroscience program."

    ``In every case the proposal succeeded or failed based on the experimental content. Even when a single modeling expert is recruited as an ad hoc reviewer, his or her opinions are overridden by the majority of study section members who know little and care less about modeling. I have seen this both as a member of the review group and as an applicant."

  • Arguments against a separate study section for mathematical biology (and for inclusion of modelers in specific study sections):

    ``The problem with a modeling study section is that the underlying biology may be equally misunderstood. It might be more to the point to try to get some modelers with expertise in a given area on regular study sections. However interdisciplinary funding has always been given more lip service than funding. "

    ``Since math biology is applied to all areas of biology, I don't believe there could be a math biology study section. The more important thing for a study section to be able to do is to evaluate how important the proposed biological problem is. I doubt whether a Math biology Study section could do this with such a wide range of proposals submitted."

  • Other suggestions:

    ``The reviews from biostatisticians were very unuseful - I strongly believe that biostatisticians should not review modelling grants."

    ``Put NIH quantitative types (not statisticians) on every study section."

    ``Experimentalists evaluating theoretical/mathematical modeling proposals use standards for experimental proposals. Could we get together and write a set of theoretical guidelines? I have some I've used in the past. I think experimentalists would largely be receptive."

    ``One possibility is to make up a data base of experts in theory that specialize in various bio-fields - and then make this data base available to the various study section. Perhaps this could be done under the auspices of the SMB but also including Soc Biomed Engineers and Biophys Soc ... "

Advice to applicants
  • ``Target the mission of NIH. Identify a problem of interest to the biological/medical community. Place the proposed work in an appropriate context, and address the anticipated impact of the work on biomedical research. Mathematics is just a methodology, just like micropuncture, histochemistry, etc, which is brought to bear on a biological problem. The study groups have little enthusiasm for mathematical novelty, but rather focus on the payoff of the application."
  • ``The narrower the proposal the better."
  • ``Present enough background to show that you are familiar with the currently literature and outstanding biological issues."
  • ``Present significant amount of work in the form of preliminary studies that clearly show the types of results that one can get from a model, and that these results are interesting."
  • ``The results should be testable, otherwise it amounts to speculation rather than science."
  • ``It is important to have experimental colleagues willing to discuss ideas and test results of the models and to have them write letters of support."
  • ``Unless a proposal is fronted by an MD or has a high experimental component, I do not expect it will have any chance for funding with the current NIH review structure."
  • ``The proposal has to be written so that the experimental biologists can understand it even if he or she cannot understand the math. Probably the best choice of getting funded is to get a special study section. In the cover letter one should emphasize tat for peer review, people with math expertise must be on the study section."
  • ``When revising a proposal: Persist and reapply to EVERY nuance of the critique. Don't fight, just argue carefully, with evidence or re-write to do what they wish."

Return to the Table of Contents