Examining Administrative Research Data to Track Postdoc Career Outcomes: a Talk by Dr. Nancy Calvin-Naylor

Examining Administrative Research Data to Track Postdoc Career Outcomes: a Talk by Dr. Nancy Calvin-Naylor

This is a guest post by Future of Research policy activist, Adriana Bankston, who moderated this plenary session at the 2017 NPA meeting.


Postdoctoral scholars make up a large segment of the biomedical workforce, and tracking their career trajectories is imperative for providing them with the most appropriate training. This information would also facilitate current postdocs to connect with former trainees who transitioned into various career paths, and thus develop a network of professionals to have as a resource throughout their careers.


Despite the importance of tracking postdoctoral career outcomes, data on this topic are difficult to obtain, and the best methods for data collection are still being debated in the scientific community. The speakers in one of the plenary sessions at the 2017 National Postdoctoral Association Annual Meeting entitled “Data driven approaches to tracking postdocs,” attempted to address some of these issues. One of the main speakers in this session was Dr. Nancy Calvin-Naylor, Managing Director at the Institute for Research on Innovation & Science (IRIS) at the University of Michigan.


What is the recent state of the biomedical research workforce?

Dr. Nancy Calvin-Naylor’s talk entitled “Examining Administrative Research Data to Track Postdoc Career Outcomes” began with the traditional definition of the postdoc experience as an apprenticeship for an independent academic research career. However, she pointed out that fewer academic research positions exist than the number of graduate students and postdocs, and trainees are now pursuing a variety of different career paths. Although a variety of approaches have been taken to their track career outcomes to date, there are still a lot of unknowns (shown below in red) about the postdoc population, from a 2012 publication1 by a commission on the state of the NIH biomedical research workforce.


Conceptual framework of the PhD Biomedical Research Workforce (from this publication)


How does IRIS operate?

The Institute for Research on Innovation & Science (IRIS) is a data repository whose purpose is to gather and curate administrative research data about the productivity and public value of the research enterprise in support of policy-making, outreach, research management, and expansion of knowledge. IRIS is a core facility at the University of Michigan, established in 2015 with 3 years seed funding for infrastructure from the Sloan & Kauffman Foundations.


The operation of IRIS is outlined in the flowcharts below. Briefly, the data provided by members (universities) is filtered through nodes in which data are de-identified, processed and secured, and approved users are then able to securely access the aggregate datasets. In this process, partners also receive data from IRIS to securely access.



Data provided by IRIS member universities (called “UMETRICS” data) are linked with various datasets held by Census as part of their Innovation Measurement Initiative. The result of this linkage is a rich view of employment and earnings of individuals employed by research grants.



Study examining employment and earnings for PhD recipients

Dr. Nancy Calvin-Naylor highlighted a study examining employment and earnings for PhD recipients. The Science paper2 from June 2016 by Nikolas Zolas et al. uses combined UMETRICS and Census data to examine 2010-2012 earnings and placement outcomes of 2009-2011 PhD recipients from a total of 8 institutions. Their results show that while the majority (57.1%) went into academia, a large percentage (38.7%) found jobs in industry. As stated in the publication, the individual earnings data were derived from links to administrative records, and resulted in de-identified versions of the UMETRICS data from IRIS. In addition, UMETRICS doctoral recipients were also placed at larger establishments, and the median establishment employing them also had a higher payroll per worker. When comparing individual earnings after one year of leaving the university by sector, those in industry earned more than those in government and academia (and academics earn less than $50,000 per year). When comparing by discipline, those in engineering earned more, while those in biology, chemistry and health disciplines again earned just under $50,000 per year, and this suggested to the authors that these individuals were likely taking postdoc positions.


Study examining gender differences in academia

The second studyhighlighted by Dr. Nancy Calvin-Naylor, published in The American Economic Review in May 2016 by Catherine Buffington et al., also utilizes a combination of UMETRICS data and Census data to examine gender differences during graduate training and for early career investigators from 4 universities over a period of 10 years. Specifically, they compared the training environments and labor market outcomes of female and male doctoral recipients. Some of the data highlighted in this talk showed that women generally earn less than men (more women were at the low-to-middle portion, and less at the higher portion) across all employment sectors, with the gap being smaller in academia and government. Importantly, and the key takeaway from the paper, is that this effect disappears when controlling for the presence of children in the household. In other words, women are disproportionately engaging in child care activities that result in these gender differences.


Using UMETRICS to assess research value and underrepresentation

Little is known about the training environment for women in STEM, as well as their career trajectories. In this regard, another study4 highlights UMETRICS as a valuable tool for quantifying both the value of research and assessing this underrepresentation. UMETRICS can also provide information on specific job titles that people hold, as well as the funding resource for research projects. These data allow the construction of networks to identify the players present in a specific team. More broadly, these data facilitate learning about the structure of the biomedical enterprise in terms training and diversity, including the training environments of women vs. men in STEM fields.


IRIS pilot studies on postdoc career outcomes and titles

IRIS is now more focused on understanding trends in postdoctoral career outcomes. In pilot studies, they will examine career outcomes for postdocs by similar methods to looking at PhD recipients. They are also currently examining the titles for postdocs paid on research grants. While details of these studies are not publicly available, discussing their limitations will be important. Since these studies use UMETRICS (administrative data), they do not take into account personal experiences or motivations of individuals. Another challenge will be matching up the years of available data between the UMETRICS and the Census information. Another consideration for IRIS is the finding the best way to maintain the privacy and confidentiality of these data, while sharing them with the scientific community.


Future directions for IRIS studies

IRIS plans to expand their dataset to include other groups, as well as invite more institutions to become IRIS members for a more complete dataset and thus covering a larger portion of the biomedical workforce. They will also release the research data in a manner that is acceptable to the participants involved, and some data are already available provided the signing of various agreements.


Note: We obtained permission from Dr. Calvin-Naylor to use these images, and had the information checked for accuracy.



  1. National Institutes of Health. (2012). Biomedical Research Workforce Working Group Report (https://acd.od.nih.gov/Biomedical_research_wgreport.pdf)
  2. Zolas N, Goldschlag N, Jarmin R, et al. Wrapping it up in a person: Examining employment and earnings outcomes for PhD recipients. Science. 2015; 350(6266): 1367-1371.
  3. Buffington C, Harris BC, Jones C, Weinberg BA. STEM Training and Early Career Outcomes of Female and Male Graduate Students: Evidence from UMETRICS Data linked to the 2010 Census. The American economic review. 2016; 106(5): 333-338.
  4. McFadden Allen, B, Lane JI, Rosen R, Owen Smith J, Weinberg, BA. UMETRICS as a Tool for Quantifying the Value of Research and Assessing Underrepresentation. AWIS Magazine. Summer 2015.

Submit a Comment

Your email address will not be published. Required fields are marked *