S3). Analysis was done at the application level with robust standard errors clustered by applicant. Institutional National Research 552a). PAR-22-180: Maximizing Investigators' Research Award (R35 - grants.nih.gov The goal of MIRA is to increase the efficiency and efficacy of NIGMS funding. Funds are generally awarded to applications with impact scores below a given percentile, which can differ both year to year and between ICs based on available funds; however, if an application is of particular relevance to the funding IC, it may still be awarded even if its score is above the typical percentile-based payline (i.e., the application receives discretionary funding). When you prepare a resubmission (A1) application, be sure to address reviewer concerns, highlighting your responses to their comments in a one-page introduction. Organizational-level controls included the applicant organizations Carnegie classification [R1: doctoral universities (highest research activity); R2: doctoral universities (higher research activity), medical school, and other], the applicant organizations type in IMPAC II (higher education, hospital, research organization, and other), and the applicant organizations geographic region as defined by the U.S. Census (northeast, midwest, south, west, and outside the United States), all treated as categorical variables. NIH: What is a "resubmission"? - Academia Stack Exchange A resubmission (A1) application may not be submitted before issuance of the summary statement from the review of the previous new application. Keep in mind you might not have a study section choice for some funding opportunity announcements, e.g., PARs. Through applicants are no longer required to mark their changes, Drs. In contrast, among applications that reach the discussion stage, controlling for topic choice substantially changes the probability of award for AA/B relative to WH scientists. In a multi-project application, you must submit an introduction with the Overall component, but introductions within the other components are optional. The next level of intervention is to develop and implement mentoring programs that provide all new and ESIs with quality guidance on navigating the NIH system. An additional application of spectral clustering was used to collapse these centroids to meta centroids, e.g., centroids of the samples. Fig. Small Business Innovation Research (SBIR)*, Commercialization Readiness Pilot (CRP) Program*, AIDS and AIDS-Related Applications Distribution of AA/B and WH PIs across institutional funding quintiles. Lets begin with looking at award rates: as a reminder, award rates are the total number of awards divided by the total number of applications. Although applicant-specific decisions about whether to resubmit may be an area for targeted intervention, those decisions do not contribute significantly to the gap in funding applications from AA/B or WH investigators. All - new, renewal, resubmission, revision, AIDS and AIDS-related, Small Business Technology Transfer (STTR)* Our analysis shows that all three of the factors that underlie the funding gappreference for some topics over others, assignment of poorer scores, and decision to discuss an applicationrevolve around decisions made by reviewers. Sometimes waiting has little impact on the timing of an award. The finding that AA/B applicants appear to be less likely to revise and resubmit has important policy implications. Responds to major weaknesses raised in the summary statement. Here, we seek to answer those questions by examining the characteristics of applications submitted by AA/B and WH scientists. This browser is not supported - Some features might not work. At the discussion stage, applications from AA/B scientists receive poorer overall impact scores on average than those of WH scientists (38.4 13.4 SD and 35.2 12.6 SD, respectively, P < 0.0001; table S4). Highlight your reviewers' comments in your introduction and the body of your application, usually in the Research Plan. Use this summary of NIH's April 21, 2021 Open Mike blog post, NIAID information, and RePORT links. ND indicates applications that were not discussed and therefore not scored. Program (LRP) Contact & Engage, NIH Office of And if you need additional preliminary data or new data is imminent, wait until you have it before resubmitting. Because of space constraints, every other cluster number is reported on the x axis; cluster numbers for the first and last eight clusters are highlighted on the graph. In total, our model accounts for 42% of the observed difference in the rates at which AA/B and WH scientists receive funding (see the Supplementary Materials for full details of regression analyses). Use the table below and the activity code (e.g., R01) specified in the title of the opportunity to determine application cycles and their relationship to due dates, review and council dates, and earliest possible start dates. Five SMEs were given a list of 10 applications from two word2vec clusters (five labeled as group A and five labeled as group B). Fig. After an unsuccessful resubmission (A1), you may submit the idea as a new application. As shown above, the number of applications to the NIGMS AREA program declined by over 40% in the past 4 years, suggesting that the decrease in NIGMS funding for the program has been primarily due to a decrease in incoming applications. A resubmission is an unfunded application that has been modified following initial review and resubmitted for consideration. Doyle J. M., Quinn K., Bodenstein Y. Word clouds are placed in a clockwise orientation relative to the order shown in Fig. by A1 scores (or byA0-to-A1 differences, in intervals like -5 to -1, 0, +1 to +5) and what is the success rate for each of those cells? The following due dates apply to New and Resubmission applications: Non-AIDS applications: January 26, 2021; May 26, 2021; September 28, 2021; January 26, 2022; May 26, 2022; September 27, 2022; January 26, 2023; May 26, 2023; September 26, 2023 Furthermore, there is no correlation between award rate and field size, as represented by the total number of applications in a cluster (fig. Keep in mind you might not have a study section choice for some funding opportunity announcements, e.g., PARs. The number of clusters per study section was thresholded using the same method. Topic choice alone accounts for over 20% of the funding gap after controlling for multiple variables, including the applicants prior achievements. Renewal/resubmission/revision and AIDS-related applications may have different due . With the text preprocessed, we trained the word2vec word embedding over the corpus using the implementation in the program gensim (36, 37). To examine how topic choice might relate to funding outcomes in general, and to the gap in funding for AA/B investigators in particular, we used word2vec (12), an informatics approach that uses word embedding of text to build document vectors suitable for grouping applications into clusters based on the similarity of their content (see Materials and Methods for details). A resubmission application can follow a competing new, renewal, or revision application (A0) that was not selected for funding (including applications "not discussed" in review). Development, Application To analyze the effect of institution type on the funding gap, we extracted all R01 applications (Types 1 and 2) in FY 20112015 from the IMPAC II database, grouped applications by institution, and then ranked institutions from the highest to lowest aggregate amount of funding. Controlling for institutional resources and career age reduces the gap in the number of submissions by 39%; when additional variables are added, the difference is no longer statistically significant (see below). Together, these three factors account for 43.2% of the modeled difference at this stage. (A) Topic clusters with the highest percentage of applications from AA/B scientists. (ORRA), Office of Laboratory Animal Welfare An analysis of both new (Type 1) and renewal (Type 2) R01 applications (N = 157,549; attributes summarized in table S1) shows that, although the award rate has dropped for all applicants over the past decade, the funding rate for WH scientists remains approximately 1.7-fold higher than for AA/B scientists [16.1% AA/B versus 29.3% WH in fiscal year (FY) 20002006 (3) and 10.7% AA/B versus 17.7% WH in FY 20112015; Fig. These results demonstrate the existence of topic preference, meaning that different topics are accorded different levels of acceptance and/or enthusiasm, which may reflect shared, broadly held views on the relative scientific value of different areas of research. Topics that focus on fundamental and mechanistic questions are distributed across the entire range of award rates (table S6). Inclusion in an NLM database does not imply endorsement of, or agreement with, When one application is funded, NIH automatically withdraws the other. Success Stories: Simphotek. When successful, this allows words that are close to each other semantically to be close to each other in the embedding space. These data are publicly available through the NIH Commons (https://era.nih.gov/), except for personal identifying information, including race, ethnicity, and sex of applicants, per NIH policy. You may be able to start revising right away, but take the time to do the best job you can. Statistically significant variation in the award rates of topic clusters. NIH separates the review process from funding decisions. 3A. To find success rate data, go to Success Rates on NIH's RePORT Web site. We used these quintiles in the rocket charts in fig. Half the reason for writing this time is to allow you a forum on our site to comment on what the new NIH resubmission policy means for the NIA community. In this way, in the preceding sentences, we would replace both NLP and natural language processing with a single token: natural_language_processing. In addition, we substituted for known terms and phrases in the National Library of Medicines Medical Subject Heading (MeSH) vocabulary. At this stage, the inclusion of topic choice as a variable in the model makes a relatively small contribution (8.2 versus 8.6 percentage point reduction, P = 0.03). We included all controls described above. Contributed to the writing/editing to the paper: T.A.H., A.L., K.A.W., R.A.M., M.J.P., A.F.D., M.S.L., H.A.V., J.M.A., and G.M.S. Some people begin revising even before getting the summary statement because waiting may cause them to miss the next receipt date. The total number of applicants with a reported race/ethnicity is 45,998. Resubmissions have a higher success rate In 2017, the overall NIH success rate for first-time Research Project Grant submissions was only 13.0% (>38,000 applications).1 Compared to 30.1% for resubmissions In 2016, the NSF received >49,000 proposals and made nearly 12,000 awards (24% funding rate).2 Resubmission success rates are higher across . Multiple registrations are required to prepare your application and apply for funding. As one high-level indicator of topic choice, we looked at applications that propose studies using human subjects, animal subjects, both, or neither and found very different results for AA/B scientists compared with scientists of other racial/ethnic groups (table S1). A Closer Look at the NIGMS AREA (R15) Program 2C). To gauge whether the study section may have been unsuitable, use the roster attached to your summary statement to view the committee members' names, and check out their publications. PIs are often surprised and disappointed (some even rant and rave) when after responding to comments in the A1, their IS declines from the A0. document.getElementById( "ak_js_1" ).setAttribute( "value", ( new Date() ).getTime() ); 9000 Rockville Pike, Bethesda, Maryland 20892, National Institutes of Health Office of Extramural Research. Before you begin a new (A0) application in the same vein as a prior application, take a hard look at whether another attempt using the same idea is likely to result in funding. This browser is not supported - Some features might not work. Hutchins B. I., Yuan X., Anderson J. M., Santangelo G. M., Relative Citation Ratio (RCR): A new metric that uses citation rates to measure influence at the article level. Health & Parenting Guide - Your Guide to Raising a Happy - WebMD Assignment of a document to a cluster was made by finding the closest meta centroid. The success rate of 42% in FY 2021 is a slight increase from that of FY 2020 (40%). The .gov means its official. S9. They made the corresponding changes are described in the introduction. Additional data from FY 20062010 Type 1 R01s were used to construct some of the control variables, as detailed below. Comparison of topic choice variation by race. S6). To rule out the possibility that separating applications into topic areas reveals a previously unidentified predictive power of percentile score, we asked whether publications resulting from R01 awards in higher- and lower-success clusters differ in their scientific influence, as measured either by the Relative Citation Ratio [RCR; an article-level metric that measures the influence of an individual publication relative to its cocitation network (21)] or by the number of raw citations they receive per year. You'll see the "MAA" (Multiple Active Application) flag in the Commons. (If youre interested in looking at new and competing renewals in aggregate, for this and the following figures, these are shown in the Excel file weve posted to the RePORT website.). The data is not completely capture submission behavior. You have no guarantee of anyone's presence at the meetingincluding past reviewersbecause participants rotate on and off, and NIH uses ad hoc reviewers. Attributes of R01 submissions by race of applicant. Time between initial application submission and resubmission. 4C). I am most grateful to my colleagues in the OER Statistical Analysis and Reporting Branch for helping put these data together. For example, you must address all your reviewers' points and highlight their comments in the introduction and usually in the Research Plan (unless too much has changed). Targeted funding opportunities such as the NIGMS MOSAIC program (34), which is designed to enhance postdoctoral career transitions to promote faculty diversity in the biomedical research workforce, may help address this. A resubmission application can follow a competing new, renewal, or revision application (A0) that was not selected for funding (including applications "not discussed" in review). Complex problems such as this are frequently studied with multivariate regression analysis, which can account for the effect of many independent variables on a single dependent variable. Presented with 10 sets of 10 applications, representing increasing degrees of semantic overlap, SMEs reproduced the groupings generated by the computational method 97.6% of the time, indicating a very high degree of correlation between word2vec and human judgment (table S5). The marked skew in topic choice by AA/B applicants led us to investigate whether those areas of science share commonalities or are instead broadly distributed across the biomedical landscape. Mathematical modeling of the NIH review process has found that subtle depressions in scorethe equivalent of a three-quarter point reduction on a scale of 1 to 9 by the three reviewers who provide the initial critiques used to inform which applications will be discussedare sufficient to substantially bias the number of funded applications in favor of a preferred class of investigators (28). Don't rush. Table S9. Submission Policies, Coronavirus Disease 2019 Notably, 37.5% of all applications from AA/B scientists mapped to only 8 of the 150 topic clusters (compared to a random distribution, P < 0.0001). Success Rates. Again, the second A0 application should follow the FOA's procedures for new applications. (B) Topic clusters with no applications from AA/B scientists. Of those eight clusters, six had award rates that were significantly below the NIH average (table S6). Specifically, we first substituted for acronyms that were defined in the text via parentheticals and present in at least five documents. As an independent method of controlling for organization-level characteristics, we used separate binary indicators for each applicant organization to more directly compare AA/B and WH applications from the same organization. To identify underlying causes of this funding gap, we analyzed six stages of the application process from 2011 to 2015 and found that disparate outcomes arise at three of the six: decision to discuss, impact score assignment, and a previously unstudied stage, topic choice. Together, these two observations suggest that AA/B scientists may be proposing to study a different distribution of topics than other applicants. But if changes are so extensive that most text would be affected, explain them in the introduction only. There is also no evidence that IC scorebased or discretionary funding decisions correlate with an applicants race. To visually compare the AA/B data with the much larger numbers in the WH dataset, we used rates per applicant (e.g., dividing the number of applications from AA/B scientists by the number of AA/B applicants). Resubmission Applications | grants.nih.gov Below, we provide NIDCR success rates for Early Stage Investigator (ESIs) and Experienced Investigator (EI) R01 applications. If your reviewers were not enthusiastic about your idea, reassess the study section. The clustering grouped areas of science in a way that was defined by the content of the applications but was agnostic to the underlying administrative boundaries. Regression analysis results for all variables. The site is secure. Warning! NIMH Career Development Programs (K-Series) To assess past success, we used continuous variables to describe both the number of prior R01 applications and awards per applicant. Select "Resubmission" in Type of Application field (box 8) on the SF424 R&R form. Questions related to a specific opportunity should be directed to the IC contact listed in Section VII of the funding opportunity. Don't hesitate to make other changes. Applicants and Recipients of NIH Funding, Supporting a Safe and Respectful Workplace, NIH Expectations, Policies, and Requirements. In a resubmission of a revision application the same introduction must describe within the standard page limit the nature and impact of the revision and summarize the changes made to the application since the last submission. The applications were clustered into three differently sized partitions, k = 30, 150, and 300. Before committing to a resubmission, you may want to assess your other Options if Your Application Isn't Funded as well. Since both the original and resubmitted application had the opportunity to be awarded, each is treated as a separate entry, with a control indicator for resubmissions. You only have one opportunity to resubmit, so make it your best shot. Controlling for an applicants prior funding history does not change this result; unfunded applications from AA/B scientists were not statistically less likely to be resubmitted than were those from WH scientists, regardless of whether the applicants were new investigators or established scientists who previously held an NIH research award (new: AA/B 50.0% versus WH 59.8%, P = 0.16; established: AA/B 56.7% versus WH 53.7%, P = 0.85; Fig. Table S6. (DBRW), Division of Human Subjects Each application is typically assigned to three reviewers who provide the initial critiques used to inform which applications will be discussed and scored by the full study section. As a library, NLM provides access to scientific literature.