By: Sara Gonzales, MS, MLIS Data Librarian

In Spring 2021, through a five-year cooperative agreement awarded by the National Library of Medicine at the NIH, Galter Library established the Network of the National Library of Medicine (NNLM) National Evaluation Center (NEC). The NEC provides evidence-based evaluation frameworks, tools, and best practices to support evaluation and continuous improvement of NNLM programs and services, including PubMed and drug terminologies, data curation and visualization, and continuing education specializations. The nationally distributed NNLM program also makes subawards to member organizations across the country, who in turn create programs and services that will benefit from the NEC’s expertise in evaluation. The NEC carries out its mission through the work of four cores:

Administration Core

• Management of network-wide evaluation data 
• Establishment of communication, process, and engagement infrastructure 
• Persona profiles to represent key NNLM audiences and stakeholders

Core Members: Matthew Carson, PhD, Head of Digital Systems at Galter Library; Sara Gonzales, MS, MLIS, Data Librarian; Roger Anderson, Director of Marketing and Communications at NUCATS; and Galter Library Developers Austin Sharp and Eric Newman

Evaluation & Continuous Improvement Core

• REDCap data capture  
• Monthly Evaluation Working Group meetings for evaluation champions across the network 
• Tableau dashboards to visualize evaluation outcomes and inform strategic decision-making 
• Quantitative and qualitative data analysis and reporting 

Core Members: Keith Herzog, NUCATS Administrative Director and Director, Trial Innovation Network Liaison Team; Emily Traw, Assistant Director of Center for Education and Career Development, NUCATS; and Pearl Go, Research Data Analyst

Community & Capacity Building Core

• Systems-informed empowerment evaluation 
• Building Communities of Practice 
• Evaluation capacity and general member surveys and key informant interviews 
• Training and other workforce development

Core Members: Gregory Phillips II, PhD, Assistant Professor of Medical Social Sciences and Preventive Medicine (Epidemiology); Leah Neubauer, EdD, MA, Associate Professor of Preventive Medicine (Public Health Practice); Amy Johnson, PhD, Research Assistant Professor of Pediatrics (Adolescent Medicine); and Esrea Perez-Bill, Data Assistant in the Evaluation, Data Integration, and Technical Assistance (EDIT) Program

Dissemination & Impact Core

• NNLM Beacon repository, featuring open and FAIR infrastructure (Findable, Accessible, Interoperable, Reusable) for the national network 
• Development of impact reports and a national data visualization challenge 
• Training development and delivery

Core Members: Karen Gutzman, MSLS, MA, Head, Research Assessment and Communications; Annette Mendoza, MLIS, and Mao Soulakis, MS, Research Assessment and Communications Librarians at Galter Library

The NEC has hit the ground running, and is pleased to report that at roughly the halfway point of Year 1 of the five year grant, we have achieved the following goals focused on internal processes, NNLM partners, and community and capacity building.

Internal Resources and Processes: 

• Successful migration of the data warehouse from the University of Washington and timely turn-around of NNLM and NIH data requests 
• Launch of NEC homepage on the NNLM website 
• Internal and external communication channels established

Support for NNLM Partners: 

• Evaluation forms updated and made available on the NUCATS REDCap 
• Evaluation Working Group launched and growing 
• Continuous improvement projects, including NLP of qualitative evaluation data with Yuan Luo, PhD, Professor of Preventive Medicine (Health and Biomedical Informatics) 
• Requirements gathering for Beacon repository with key stakeholders

Community & Capacity Building: 

• Enhancement of NNLM evaluation resources  
• NNLM Member survey coordination and oversight 
• Highlight underrepresented voices and evaluation needs 
• Training and engagement evaluation support and development of training opportunities

The team members of the NEC bring decades of combined experience in evaluation, capacity building, and continuous improvement. If you would like to learn more, connect with us! Join the NEC newsletter, keep an eye out for NEC trainings hosted on the NNLM site, and watch for updates to our NEC webpage. Please contact NEC Director, Kristi Holmes, PhD, Director of Galter Library and Professor of Preventive Medicine (Health and Biomedical Informatics) with additional questions.