The Enhancing Public Health Laboratory Capabilities and Increasing Capacity cooperative agreement between APHL and the US Centers for Disease Control and Prevention (CDC) includes more than 600 activities that span every program in APHL and across eight centers within CDC. In the course of monitoring and evaluating those activities, APHL has collected information on more than 1,000 indicators associated with 200 unique measures.
That body of data represents a rich opportunity to highlight the value and impact of the programs for member laboratories and public health broadly. “We would like to be able to tell the stories of all the good work that is being done with these funds,” said Eric Blank, DrPH, APHL’s principal investigator on the cooperative agreement. “Those stories are in the numbers; we just have to pull them out.”
APHL and CDC’s Division of Laboratory Systems (DLS) have been working in close partnership to design and build a new centralized monitoring and evaluation system to assess and report the public health impact of these programs. Teams are working to select metrics, streamline data collection, facilitate analyses and display outcomes.
“This process brings us back to the purpose and intention of our cooperative agreement—to strengthen and enhance the effectiveness of public health labs, both individually and as components of a larger public health system,” said Collette Fitzgerald, PhD, focus area coordinator for foundational leadership and support for the cooperative agreement and deputy director for science in DLS.
The project team started by gathering intended users of the monitoring and evaluation data and coming to consensus on what measures to assess, standard definitions for each and how to report them. “It has been a huge undertaking,” said Lorelei Kurimski, MS, APHL’s director of Quality Systems and Analytics. “The partnership and collaboration with CDC have been crucial throughout this whole process.”
It has also been an opportunity to step back and look at the big picture of what’s working well and what isn’t, Blank said. “You must be asking the right questions and collecting the right information from the beginning. That will allow us to show what we’ve accomplished, and how we know,” he said. “We’re not done yet, but we are making progress.”
Some measures are used by programs for assessing activities within their projects as part of cooperative agreement; others are collected to help guide project development. In 2022, the APHL Monitoring and Evaluation team handled 25 data requests and supported 51 surveys for program gap analyses and needs assessments, collecting more than 140,000 data points.
“We can learn a lot through this process, but the goal is not just lessons learned,” Kurimski said. “We want to apply those data to inform continuous quality improvement for programs moving forward.”
To make the data broadly accessible, the team is also developing a series of evaluation briefs and infographics to present information in a digestible way. Interactive dashboards allow users to visualize and drill into the data to answer specific questions. Within the dashboard based on APHL’s 2022 Workforce Profiles Survey, for example, users can look at data from targeted regions or compare it against national benchmarks. What are typical salary levels in a particular geographic region? What types of benefits, job factors and culture are important for public health laboratorians’ job satisfaction?
“With data science, we can use statistics, computer science and machine learning to really gain insights into the data,” Kurimski said. “We want to get our members and CDC partners data they can use to help inform their decisions and their decision-makers.”
RELATED LINKS