Monitoring & Evaluating Programs

1. Health centers can monitor and evaluate (M&E) their programs to assess progress toward programmatic goals, identify challenges, and facilitate program adaptation to improve service delivery and reach the desired outcomes.

• This module developed by the World Health Organization (WHO) provides information on how to monitor and evaluate the performance of PrEP programs, and how health providers can use actionable data for decision-making and quality improvement.
  • World Health Organization. WHO Implementation tool for pre-exposure prophylaxis (PrEP) of HIV infection: module 5: monitoring and evaluation. World Health Organization; 2018.
    
  • BLUPrInt has developed a set of Quality Assurance Metrics that can be integrated into PrEP program evaluation.
    

2. A logic framework is a useful tool that health centers can use to map out available resources and identify the objectives of a program and how they will be measured. The tool can help program implementers articulate what the inputs, outputs, outcome, and impact of the program are, and specify indicators for M&E activities.

• This presentation discusses how implementation science can be harnessed in the Ending the HIV Epidemic (EHE) initiative to inform tools, models and frameworks. When programs use a logic framework that specifies intervention components, articulates mechanisms of action, and details strategies and outcomes at the provider, program, and system levels.
  • Golub SA. Bridging the Gap: Implementation Science as an Implementation Strategy for EHE. PSMG: Innovations in Ending the HIV Epidemic Series; November 2020.7
    
    
• This paper describes the Implementation Research Logic Model, a tool designed for implementation scientists and practitioners to enhance the rigor and transparency of describing the often-complex processes of improving the adoption of evidence-based interventions in healthcare delivery systems.”
  • Smith JD, Li DH, Rafferty MR. The Implementation Research Logic Model: a method for planning, executing, reporting, and synthesizing implementation projects. Implementation Science. 2020 Dec;15(1):1-2.7
    

3. Health centers can use the PrEP cascade to quantify steps and identify gaps in service delivery. Mapping real-world programmatic / clinical data onto the PrEP cascade can help implementers monitor program performance and inform strategies to improve service delivery.

• This paper presents evidence about how the PrEP cascade can be used to guide the planning, implementation, and monitoring performance of real-world PrEP programs. Using routine programmatic data to populate the PrEP cascade, the authors identified missed opportunities in PrEP implementation, and employed interventions to increase effective and equitable delivery of PrEP services.
  • Were D, Musau A, Mutegi J, Ongwen P, Manguro G, Kamau M, Marwa T, Gwaro H, Mukui I, Plotkin M, Reed J. Using a HIV prevention cascade for identifying missed opportunities in PrEP delivery in Kenya: results from a programmatic surveillance study. Journal of the International AIDS Society. 2020 Jun;23:e25537.7
    

4. A good M&E plan should be data-driven and minimize the data collection and reporting burden. Extracting data from the electronic medical record (EMR) is a good start but may prove challenging for capturing process metrics.

• This paper presents the minimum reporting variables (n=4) needed for ongoing monitoring of PrEP programs and inter-country comparison. It utilizes routine surveillance items to monitor key stages in PrEP delivery and to inform strategies to improve program performance.
  • Saunders J, Gill ON, Delpech V, Estcourt C. Minimum dataset for monitoring national human immunodeficiency virus pre-exposure prophylaxis (HIV PrEP) programmes: a five-nation consensus, 2019. Eurosurveillance. 2021 Jun 10;26(23):2001595.]7
    
    
• These papers demonstrate how EMR data can be useful for the monitoring and evaluation of HIV treatment and prevention programs, while also noting the need to refine existing EMR system to allow the collection of additional M&E metrics (e.g., process indicators).
  • Reidy WJ, Rabkin M, Syowai M, Schaaf A, El-Sadr WM. Patient-level and program-level monitoring and evaluation of differentiated service delivery for HIV: a pragmatic and parsimonious approach is needed. AIDS (London, England). 2018 Jan 28;32(3):399.]7
    
  • Nash D, Elul B, Rabkin M, Tun M, Saito S, Becker M, Nuwagaba-Biribonwoha H. Strategies for more effective monitoring and evaluation systems in HIV programmatic scale-up in resource-limited settings: implications for health systems strengthening. JAIDS Journal of Acquired Immune Deficiency Syndromes. 2009 Nov 1;52:S58-62.]7
    

5. Significant disparities exist in PrEP awareness, uptake, and HIV incidence in the U.S. M&E metrics measuring disparities are needed to ensure equity.

• These papers present evidence of the racial and ethnic disparities in PrEP awareness and uptake in the U.S.
  • Nosyk B, Krebs E, Zang X, Piske M, Enns B, Min JE, Behrends CN, Del Rio C, Feaster DJ, Golden M, Marshall BD. “Ending the Epidemic” Will Not Happen Without Addressing Racial/Ethnic Disparities in the United States Human Immunodeficiency Virus Epidemic. Clinical Infectious Diseases. 2020 Dec 1;71(11):2968-71.
    
  • Kanny D, Jeffries IV WL, Chapin-Bardales J, Denning P, Cha S, Finlayson T, Wejnert C, Abrego M, Al-Tayyib A, Anderson B, Barak N. Racial/ethnic disparities in HIV preexposure prophylaxis among men who have sex with men—23 urban areas, 2017. Morbidity and Mortality Weekly Report. 2019 Sep 20;68(37):801.
    
  • Elopre L, Kudroff K, Westfall AO, Overton ET, Mugavero MJ. The right people, right places, and right practices: Disparities in PrEP access among African American men, women and MSM in the Deep South. Journal of acquired immune deficiency syndromes (1999). 2017 Jan 1;74(1):56.
    
    
• This paper highlights the importance of including disparities metrics as part of the monitoring system in the “End the HIV Epidemic” (EtHE) plan and discusses how the health disparities can be addressed via the EtHE plan.
  • Nash D. Designing and disseminating metrics to support jurisdictional efforts to end the public health threat posed by HIV epidemics. American journal of public health. 2020 Jan;110(1):53-7.

6. Using unified metrics for M&E facilitates consistency and comparability across programs.

• This conference presentation shows how the PrEP retention outcomes varied depending on the definition of the measure. It demonstrates how the generalizability and the sharing of best practices is challenged by a lack of consistent and standardized set of measures.
  • Jason Zucker. Moving From Uptake to Engagement: Understanding Persistence in Comprehensive HIV Prevention Care. IAPAC 2020.