Rapid Response Service

Approaches for front-line organizations to implement evidence-based interventions

Download pdf

Questions

  • Are there guidelines that exist to support integration of evidence-based interventions (EBIs) into HIV service delivery?

 Key take-home messages

  • Helping community-based organizations (CBOs) find and use research evidence for the development of programs, services and advocacy is only part of the process of supporting evidence-informed front-line programs and services as there is also a need to determine how to implement and scale-up the findings into practice.
  • The articles we identified for this rapid response focused on the approach to supporting the implementation of evidence-based interventions through the Diffusion of Effective Behavioural Intervention (DEBI) program from the Centres for Disease Control (CDC) in the United States.
  • The foundation of the DEBI program is that evidence-based interventions are a necessary component of the programming for CBOs and other service provision organizations. After the CDC identifies evidence-based interventions, they are packaged into the Replicating Effective Programs (REP) project and then disseminated through the Diffusion of Effective Behavioral Interventions (DEBI) project.
  • As part of the broader DEBI program, the Mpowerment Project Technology Exchange System (MPTES) was designed for the specific purpose for helping CBOs implement interventions from the Mpowerment project. This process has been recently profiled and evaluated through interviews and pilot tests with CBOs. A series of ‘lessons learned’ from this process are outlined. In general, the CBOs involved in the project overall gave positive feedback about the MPTES but many expressed the need for it to include greater focus on diversity issues, include descriptions of how the intervention has been implemented by others and provide guidance about the ways in which the intervention can be adapted for different populations.

The issue and why it’s important

Increased attention has been paid to helping community-based organizations find and use research evidence and to understand the factors that influence their use of research evidence.(1-4) In the HIV sector in Canada, several organizations such as the Ontario HIV Treatment Network and the Canadian AIDS Treatment Information Exchange have dedicated resources and programs for helping community-based organizations to efficiently find research evidence. However, helping CBOs find and use research evidence for the development of programs, services and advocacy is only part of the process of supporting evidence-informed front-line programs and services as there is also a need to determine how to implement and scale-up the findings into practice. Surveys of community-based organizations in the Canadian HIV sector of their capacity to acquire, assess, adapt and apply research evidence indeed point to a gap in this area with capacity to adapt and apply findings being the lowest rated areas of capacity among organizations.(5;6)

In the context of this rapid response, the Women’s Health in Women’s Hands (WHIWH) community health centre (an organization that provides care to approximately 400 women living with HIV) has developed and pilot tested an intervention, designed to support African and Caribbean women going through the process of disclosing their HIV status by integrating the process into existing support and care programs. Having developed the intervention, WHIWH is now undertaking a process to develop and pilot test mechanisms and guidelines for integrating this evidence-based disclosure model/intervention and support effective rollout within community-based HIV/AIDS organizations and other service providers. To inform this process, WHIWH requested research evidence about other efforts designed to support the implementation and scale-up of interventions in community-based organizations.

What we found

The articles we identified for this rapid response focused on the approach to supporting the implementation of evidence-based interventions through the Diffusion of Effective Behavioural Intervention (DEBI) program from the Centres for Disease Control (CDC) in the United States. The foundation of the DEBI program is that evidence-based interventions are a necessary component of the programming for CBOs and other service provision organizations.(7) After the CDC identifies evidence-based interventions they are packaged into the Replicating Effective Programs (REP) project and then disseminated through the Diffusion of Effective Behavioral Interventions (DEBI) project.(7)

The interventions in REP and disseminated through the DEBI program have as their goal a reduction in high-risk sexual behaviour among people living with HIV/AIDS. In order for CBOs to most effectively implement these interventions, they are organized into user-friendly packages and written in plain language. The packages, field-tested by researchers beforehand, are meant to offer a plan for organizations to implement risk-reduction programs at their own facilities.(8) In general, REP packages are designed for specific target populations and therefore may be delivered in a variety of settings that include CBOs as well as health clinics, shelters, bars, and other locations where the intervention will be most effective.(8) The number of people involved with the implementation of each intervention also differs according to context with some being one-on-one while others are delivered in groups.(8)

The role of DEBI in implementing these prevention interventions is providing “intervention resources, training, technical assistance, and capacity-building activities on [evidence-based interventions] to health departments, CBOs, and medical providers.”(9) The goal is to enhance the capacity of these organizations to provide EBIs to their clients. Of the barriers faced by CBOs in implementing components of DEBI, a lack of technical expertise was found to be one of the most common with other challenges including the difficulty in assigning certain EBIs to certain agencies as agency capacity and client populations differ across organizations.(9)

The processes for supporting the implementation of one of the interventions included in the DEBI program – the Mpowerment project – has recently been profiled and evaluated and may provide helpful insights for others undertaking similar processes. The Mpowerment project is a community-level, evidence-based HIV prevention intervention for young gay/bisexual men that draws on a mix of HIV prevention approaches (e.g., small groups, community outreach, publicity, drop-in centres, and community mobilization).(10) As Kegeles et al. (2012) outline, the Mpowerment Project Technology Exchange System (MPTES) was designed for the specific purpose of helping CBOs implement interventions from the project. Based on experience with supporting the implementation of the Mpowerment Project in CBOs, a series of meetings held with CBOs focused on feedback about implementing the project and a pilot collaboration with one CBO, a series of ‘lessons learned’ for developing an implementation intervention for CBOs were documented. The key points from these ‘lessons learned’ include (* note that these points have been extracted directly from the article):(10)

General issues

  • written materials and limited training and TA are insufficient for effective translation of evidence-based interventions into practice;
  • while CBOs need information about the intervention’s core elements, they also need information about a myriad of other issues that affect implementation (e.g., finding space for the project, integrating it with other programs offered, finding ideal staff and how to evaluate it);
  • it is important to consider whom to intervene with at CBOs (i.e., focus on staff who will be implementing the project and tailor training materials to them);
  • CBOs need advice about supervision, selection and retention of staff members, as well as preparation and planning for staff turnover;
  • encouraging diffusion and communication across organizations is helpful and desired by CBOs (i.e., learning from other CBOs about how the intervention is being implemented in their community);
  • CBO staff do not always think through the logic of their programs (i.e., program activities do not always match stated goals, requiring a clear logic and easy to follow logic model);
  • organizations that request information about the program are at different stages of implementation (i.e., materials and strategies to assist CBOs considering implementation of the program should be developed for organizations at different stages);
  • funders often want CBOs to conduct evaluations of the program (i.e., CBOs desire and need help with developing feasible approaches to evaluation);
  • when an intervention is relatively unscripted, it is essential to provide abundant examples of how to conduct core elements (i.e., highlighting the ways in which the core elements of the intervention can be operationalized); and
  • community issues may have an enormous effect on program implementation (e.g., the sociopolitical context) and the challenges these pose need to be identified and carefully considered)

Materials

  • all materials should help the project “come alive,” and convey the essence, excitement, and dynamism of the program (i.e., CBOs should be able to ‘see themselves’ and their target community in the materials);
  • written materials need to depict and fully describe how the program should be implemented;
  • written materials should be durable and inexpensive to allow them to be easily replaced given high staff turnover; and
  • make provisions for visual learning styles (e.g., graphics and photos) to make the materials more interesting and compelling

Technical assistance

  • the most effective interactions between the TA and training providers and the CBOs is one of “exchange” rather than the “top-down” approach (i.e., implementation is a ‘two-way’ street between the CBOs and those providing technical assistance);
  • need to provide proactive technical assistance given that organizations often delay in requesting it;
  • building and maintaining rapport between CBO staff and technical assistance providers is imperative for developing trust; and
  • CBO staff do not want to be told what to do by someone outside their organization (i.e., need to support them in critically analyzing and identifying how to adapt it themselves)

Training

  • training should involve active-learning methods (i.e., individuals should get an experience of how the program “feels” in the training suing interactive approaches; and
  • organizations want to learn from each other, not solely from trainers (i.e., organizations need opportunity to share experiences and problem-solve together).

Kegeles et al. (2012) also evaluated CBOs’ uptake, utilization and perceptions of the MPTES components and issues that arose during technical assistance.(10) Based on two years of follow-up with 49 CBOs, it was found that the program manual developed for the MPTES was widely used but that while other program materials were used early in the implementation process, their use declined over time. However, after the provision of technical assistance for implementing the intervention, the usage of MPTES materials and request for technical assistance became consistent over time. In addition, the CBOs involved in the project overall gave positive feedback about the MPTES but many expressed the need for it to include greater focus on diversity issues, include descriptions of how the intervention has been implemented by others and provide guidance about the ways in which the intervention can be adapted for different populations.(10)

What we did

To identify literature for this summary, we searched for systematic reviews using Health Systems Evidence by searching the categories related to availability of care (under ‘Delivery arrangements’ and ‘How care is designed to consumers’ needs) and organization-targeted implementation strategies. We didn’t find any systematic reviews related to implementation considerations specifically for community-based organizations but two reviews (11;12) may provide some more general background for those that are interested. Next, we reviewed literature in PubMed that we retrieved using a combination of relevant terms (implementing evidence AND HIV). Lastly, we conducted a related articles search using the article by Kegeles et al. (2012) that we identified as being relevant.(10)

Factors that may impact local applicability

In interpreting the information presented in this summary, the main factor that may affect local applicability is that the DEBI program is based on the context of the United States and for a specific set of evidence-based interventions. The difference in context and in the types of interventions should accounted for in considering the development of a process of supporting the implementation other evidence-based interventions.

Reference list

  1. Wilson MG, Lavis JN, Travers R, Rourke SB. Community-based knowledge transfer and exchange: Helping community-based organiza-tions link research to action. Imple-mentation Science 2010;5(1):33.
  2. Wilson MG, Lavis JN. Community-based organizations and how to support their use of systematic reviews: A qualitative study. Evi-dence & Policy 2011;7(4):449-69.
  3. Kothari A, Armstrong R. Community-based knowledge translation: Unexplored opportunities. Imple-mentation Science 2011;6(1):59.
  4. Jack SM, Dobbins M, Sword W, Novotna G, Brooks S, Lipman EL et al. Evidence-informed decision-making by professionals working in addiction agencies serving women: A descriptive qualitative study. Substance Abuse Treatment, Pre-vention, and Policy 2011;6:29.
  5. Wilson MG, Rourke SB, Lavis JN, Bacon J, Tremblay G, Arthur J. Mo-bilizing Evidence at the Front Lines: Improving HIV/AIDS Policy and Care in Canada (draft technical report). Toronto, Canada: Ontario HIV Treatment Network; 2010.
  6. Wilson MG, Rourke SB, Lavis JN, Bacon J, Travers R. Community Capacity to Acquire, Assess, Adapt and Apply Research Evidence: A Survey of Ontario’s HIV/AIDS Sec-tor. Implementation Science 2011;6(1):54.
  7. McKleroy VS, Galbraith JS, Cum-mings B, Jones P, Harshbarger C, Collins C et al. Adapting evidence-based behavioral interventions for new settings and target popula-tions. AIDS Education and Preven-tion 2006;18(4 Suppl A):59-73.
  8. Centers for Disease Control and Prevention. Replicating Effective Programs Plus. Department of Health and Human Services, Cen-ters for Disease Control and Pre-vention 2012;Available from: URL: http://www.cdc.gov/hiv/topics/prev_prog/rep/index.htm
  9. Collins CB, Jr., Hearn KD, Whittier DN, Freeman A, Stallworth JD, Phields M. Implementing packaged HIV-prevention interventions for HIV-positive individuals: considerations for clinic-based and community-based interventions. Public Health Reports 2010;125(Suppl 1):55-63.
  10. Kegeles SM, Rebchook G, Pollack L, Huebner D, Tebbetts S, Hamiga J et al. An intervention to help com-munity-based organizations imple-ment an evidence-based HIV pre-vention intervention: The Mpower-ment Project technology exchange system. American Journal of Com-munity Psychology 2012;49(1-2):182-98.
  11. Greenhalgh T, Robert G, Macfar-lane F, Bate P, Kyriakidou O. Diffu-sion of innovations in service or-ganizations: Systematic review and recommendations. Milbank Quar-terly 2004;82(4):581-629.
  12. Kaplan HC, Brady PW, Dritz MC, Hooper DK, Linam WM, Froehle CM et al. The influence of context on quality improvement success in health care: A systematic review of the literature. Milbank Quarterly 2010;88(4):500-59.

Suggested citation

Rapid Response Service. Approaches for Front-Line Organizations to Implement Evidence-Based Interventions. Toronto, ON: Ontario HIV Treatment Network; December, 2012.

Prepared by

Michael G. Wilson and Jamie McCallum.

WordPress Image Lightbox Plugin