Skip to main content
Top

06-01-2018 | Colorectal cancer | Article

A systematic review of clinic and community intervention to increase fecal testing for colorectal cancer in rural and low-income populations in the United States – How, what and when?

Journal: BMC Cancer

Authors: Melinda M. Davis, Michele Freeman, Jackilen Shannon, Gloria D. Coronado, Kurt C. Stange, Jeanne-Marie Guise, Stephanie B. Wheeler, David I. Buckley

Publisher: BioMed Central

Abstract

Background

Interventions to improve fecal testing for colorectal cancer (CRC) exist, but are not yet routine practice. We conducted this systematic review to determine how implementation strategies and contextual factors influenced the uptake of interventions to increase Fecal Immunochemical Tests (FIT) and Fecal Occult Blood Testing (FOBT) for CRC in rural and low-income populations in the United States.

Methods

We searched Medline and the Cochrane Library from January 1998 through July 2016, and Scopus and clinicaltrials.gov through March 2015, for original articles of interventions to increase fecal testing for CRC. Two reviewers independently screened abstracts, reviewed full-text articles, extracted data and performed quality assessments. A qualitative synthesis described the relationship between changes in fecal testing rates for CRC, intervention components, implementation strategies, and contextual factors. A technical expert panel of primary care professionals, health system leaders, and academicians guided this work.

Results

Of 4218 citations initially identified, 27 unique studies reported in 29 publications met inclusion criteria. Studies were conducted in primary care (n = 20, 74.1%), community (n = 5, 18.5%), or both (n = 2, 7.4%) settings. All studies (n = 27, 100.0%) described multicomponent interventions. In clinic based studies, components that occurred most frequently among the highly effective/effective study arms were provision of kits by direct mail, use of a pre-addressed stamped envelope, client reminders, and provider ordered in-clinic distribution. Interventions were delivered by clinic staff/community members (n = 10, 37.0%), research staff (n = 6, 22.2%), both (n = 10, 37.0%), or it was unclear (n = 1, 3.7%). Over half of the studies lacked information on training or monitoring intervention fidelity (n = 15, 55.6%).

Conclusions

Studies to improve FIT/FOBT in rural and low-income populations utilized multicomponent interventions. The provision of kits through the mail, use of pre-addressed stamped envelopes, client reminders and in-clinic distribution appeared most frequently in the highly effective/effective clinic-based study arms. Few studies described contextual factors or implementation strategies. More robust application of guidelines to support reporting on methods to select, adapt and implement interventions can help end users determine not just which interventions work to improve CRC screening, but which interventions would work best in their setting given specific patient populations, clinical settings, and community characteristics.

Trial registration

In accordance with PRISMA guidelines, our systematic review protocol was registered with PROSPERO, the international prospective register of systematic reviews, on April 16, 2015 (registration number CRD42015019557).
Literature
2.
Whitlock EP, Lin JS, Liles E, Beil TL, Fu R. Screening for colorectal cancer: a targeted, updated systematic review for the U.S. preventive services task force. Ann Intern Med. 2008;149(9):638–58.CrossRefPubMed
3.
US Preventive Services Task Force. Final Update Summary: Colorectal Cancer: Screening. U.S. Preventive Services Task Force. June 2016. https://​www.​uspreventiveserv​icestaskforce.​org/​Page/​Document/​UpdateSummaryFin​al/​colorectal-cancer-screening2. Accessed Nov 14 2017.
4.
Lieberman D, Ladabaum U, Cruz-Correa M, Ginsburg C, Inadomi JM, Kim LS, Giardiello FM, Wender RC. Screening for colorectal cancer and evolving issues for physicians and patients: a review. JAMA. 2016;316(20):2135–45.CrossRefPubMed
5.
American Cancer Society. Colorectal Cancer Facts & Figures 2014–2016. Vol. 2014. Atlanta: American Cancer Society; 2014.
6.
7.
National Colorectal Cancer Roundtable. 80% by 2018. http://​nccrt.​org/​what-we-do/​80-percent-by-2018/​. Accessed 27 Nov 2017.
8.
Healthy People 2020. 2020 Topics & Objectives: Cancer. https://​www.​healthypeople.​gov/​2020/​topicsobjectives​/​topic/​cancer/​objectives#4054. Accessed Nov 27 2017.
9.
Cole AM, Jackson JE, Doescher M. Urban–rural disparities in colorectal cancer screening: cross-sectional analysis of 1998–2005 data from the centers for disease Control's behavioral risk factor surveillance study. Cancer Med. 2012;1(3):350–6.CrossRefPubMedPubMedCentral
10.
Tammana VS, Laiyemo AO. Colorectal cancer disparities: Issues, controversies and solutions. World J Gastroenterol. 2014;20(4):869–76.CrossRefPubMedPubMedCentral
11.
Klabunde CN, Joseph DA, King JB, White A, Plescia M. Vital signs: colorectal cancer screening test use - United States, 2012. SEARCH Morbidity and Mortality Weekly Report (MMWR). 2013;62(44):881–888. https://​www.​cdc.​gov/​mmwr/​preview/​mmwrhtml/​mm6244a4.​htm.
12.
Davis MM, Renfro S, Pham R, Hassmiller Lich K, Shannon J, Coronado GD, Wheeler SB. Geographic and population-level disparities in colorectal cancer testing: a multilevel analysis of Medicaid and commercial claims data. Prev Med. 2017;101:44–52.CrossRefPubMed
13.
Steele CB, Rim SH, Joseph DA, King JB, Seeff LC. Centers for disease C, prevention: colorectal cancer incidence and screening - United States, 2008 and 2010. Morb Mortal Wkly Rep Surveill Summ. 2013;62(Suppl 3):53–60.
14.
Holden DJ, Jonas DE, Porterfield DS, Reuland D, Harris R. Systematic review: enhancing the use and quality of colorectal cancer screening. Ann Intern Med. 2010;152(10):668–76.CrossRefPubMed
15.
Brouwers M, De Vito C, Bahirathan L, Carol A, Carroll J, Cotterchio M, Dobbins M, Lent B, Levitt C, Lewis N, et al. What implementation interventions increase cancer screening rates? A systematic review. Implement Sci. 2011;6(1):111.CrossRefPubMedPubMedCentral
16.
Sabatino SA, Lawrence B, Elder R, Mercer SL, Wilson KM, DeVinney B, Melillo S, Carvalho M, Taplin S, Bastani R, et al. Effectiveness of interventions to increase screening for breast, cervical, and colorectal cancers: nine updated systematic reviews for the guide to community preventive services. Am J Prev Med. 2012;43(1):97–118.CrossRefPubMed
17.
The Community Guide. Cancer Screening: Multicomponent Interventions - Colorectal Cancer. August 2016. https://​www.​thecommunityguid​e.​org/​findings/​cancer-screening-multicomponent-interventions-colorectal-cancer. Accessed 27 Nov 2017.
18.
Rojas Smith L, Ashok M, Dy SM, Wines RC, Teixeira-Poit S. Contextual frameworks for research on the implementation of complex system interventions. Rockville, MD: Agency for Healthcare Research and Quality; 2014.
19.
Yoong SL, Clinton-McHarg T, Wolfenden L. Systematic reviews examining implementation of research into practice and impact on population health are needed. J Clin Epidemiol. 2015;68(7):788–91.CrossRefPubMed
20.
Schreuders EH, Grobbee EJ, Spaander MC, Kuipers EJ. Advances in fecal tests for colorectal cancer screening. Curr Treat Options Gastroenterol. 2016;14(1):152–62.CrossRefPubMedPubMedCentral
21.
Robertson DJ, Lee JK, Boland CR, Dominitz JA, Giardiello FM, Johnson DA, Kaltenbach T, Lieberman D, Levin TR, Rex DK. Recommendations on fecal immunochemical testing to screen for colorectal Neoplasia: a consensus statement by the US multi-society task force on colorectal cancer. Gastroenterology. 2017;152(5):1217–37. e1213CrossRefPubMed
22.
Allison JE, Fraser CG, Halloran SP, Young GP. Population screening for colorectal cancer means getting FIT: the past, present, and future of colorectal cancer screening using the fecal immunochemical test for hemoglobin (FIT). Gut Liver. 2014;8(2):117–30.CrossRefPubMedPubMedCentral
23.
Tinmouth J, Lansdorp-Vogelaar I, Allison JE. Faecal immunochemical tests versus guaiac faecal occult blood tests: what clinicians and colorectal cancer screening programme organisers need to know. Gut. 2015;64(8):1327–37.CrossRefPubMed
24.
World Endoscopy Organization (WOE) Colorectal Cancer Screening Expert Working Group. FIT for Screening. 2017. http://​www.​worldendo.​org/​about-us/​committees/​colorectal-cancer-screening/​ccs-testpage2-level4/​fit-for-screening. Accessed 27 Nov 2017.
25.
Higgins JPT, Green S, Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated march 2011]. The Cochrane Collaboration, 2011. http://​handbook-5-1.​cochrane.​org/​. Accessed 27 Nov 2017.
26.
Berkman ND, Lohr KN, Ansari M, McDonagh M, Balk E, Whitlock E, Reston J, Bass E, Butler M, Gartlehner G et al: Grading the Strength of a Body of Evidence When Assessing Health Care Interventions for the Effective Health Care Program of the Agency for Healthcare Research and Quality: An Update. Methods Guide for Comparative Effectiveness Reviews (Prepared by the RTI-UNC Evidence-based Practice Center under Contract No. 290–2007-10056-I). AHRQ Publication No. 13(14)-EHC130-EF. Rockville, MD: Agency for Healthcare Research and Quality. November 2013. https://​www.​ncbi.​nlm.​nih.​gov/​books/​NBK174881/​pdf/​Bookshelf_​NBK174881.​pdf. Accessed 27 Nov 2017.
27.
Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Ann Intern Med. 2009;151(4):W65–94.CrossRefPubMed
28.
Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Rev. 2015;4(1):1.CrossRef
29.
Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. Br Med J. 2015;349
30.
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.CrossRefPubMedPubMedCentral
31.
Guise J-M, Chang C, Viswanathan M, Glick S, Treadwell J, Umscheid CA, Whitlock E, Fu R, Berliner E, Paynter R, et al. Agency for Healthcare Research and Quality evidence-based practice center methods for systematically reviewing complex multicomponent health care interventions. J Clin Epidemiol. 2014;67(11):1181–91.CrossRefPubMed
32.
Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, Proctor EK, Kirchner JE. A refined compilation of implementation strategies: results from the expert recommendations for implementing change (ERIC) project. Implement Sci. 2015;10:21.CrossRefPubMedPubMedCentral
33.
Bunger AC, Powell BJ, Robertson HA, MacDowell H, Birken SA, Shea C. Tracking implementation strategies: a description of a practical approach and early findings. Health Res Policy Syst. 2017;15(1):15.CrossRefPubMedPubMedCentral
34.
Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. 2013;8:139.CrossRefPubMedPubMedCentral
35.
Tomoaia-Cotisel A, Scammon DL, Waitzman NJ, Cronholm PF, Halladay JR, Driscoll DL, Solberg LI, Hsu C, Tai-Seale M, Hiratsuka V, et al. Context matters: the experience of 14 research teams in systematically reporting contextual factors important for practice change. Ann Fam Med. 2013;11(Suppl 1):S115–23.CrossRefPubMedPubMedCentral
36.
Stange K: An overall approach to assessing important contextual factors. Edited by Davis M; 2017.
37.
Stange KC, Glasgow RE. Considering and reporting important contextual factors in research on the patient-centered medical home. Agency for Healthcare Research and Quality: Rockville; 2013.
38.
National Heart Lung and Blood Institute. Quality Assessment Tool for Before-After (Pre-Post) Studies With No Control Group. US Department of Health & Human Services, National Institutes of Health. March 2014. https://​www.​nhlbi.​nih.​gov/​health-pro/​guidelines/​in-develop/​cardiovascular-risk-reduction/​tools/​before-after. Accessed 27 Nov 2017.
39.
Wells GA, Shea B, O'Connell D, Peterson J, Welch V, Losos M, Tugwell P: The Newcastle-Ottawa scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. 2014. http://​www.​Ohri.​Ca/​programs/​clinical_​epidemiology/​oxford.​Asp. Accessed 27 Nov 2017.
40.
Baker DW, Brown T, Buchanan DR, Weil J, Balsley K, Ranalli L, Lee JY, Cameron KA, Ferreira MR, Stephens Q, et al. Comparative effectiveness of a multifaceted intervention to improve adherence to annual colorectal cancer screening in community health centers: a randomized clinical trial. JAMA Intern Med. 2014;174(8):1235–41.CrossRefPubMed
41.
Braun KL, Fong M, Kaanoi ME, Kamaka ML, Gotay CC. Testing a culturally appropriate, theory-based intervention to improve colorectal cancer screening among native Hawaiians. Prev Med. 2005;40(6):619–27.CrossRefPubMedPubMedCentral
42.
Campbell MK, James A, Hudson MA, Carr C, Jackson E, Oakes V, Demissie S, Farrell D, Tessaro I. Improving multiple behaviors for colorectal cancer prevention among african american church members. Health Psychol. 2004;23(5):492–502.CrossRefPubMed
43.
Coronado GD, Golovaty I, Longton G, Levy L, Jimenez R. Effectiveness of a clinic-based colorectal cancer screening promotion program for underserved Hispanics. Cancer. 2011;117(8):1745–54.CrossRefPubMed
44.
Davis T, Arnold C, Rademaker A, Bennett C, Bailey S, Platt D, Reynolds C, Liu D, Carias E, Bass P 3rd, et al. Improving colon cancer screening in community clinics. Cancer. 2013;119(21):3879–86.CrossRefPubMedPubMedCentral
45.
Friedman LC, Everett TE, Peterson L, Ogbonnaya KI, Mendizabal V. Compliance with fecal occult blood test screening among low-income medical outpatients: a randomized controlled trial using a videotaped intervention. J Cancer Educ. 2001;16(2):85–8.PubMed
46.
Goldberg D, Schiff GD, McNutt R, Furumoto-Dawson A, Hammerman M, Hoffman A. Mailings timed to patients’ appointments: a controlled trial of fecal occult blood test cards. Am J Prev Med. 2004;26(5):431–5.CrossRefPubMed
47.
Goldman SN, Liss DT, Brown T, Lee JY, Buchanan DR, Balsley K, Cesan A, Weil J, Garrity BH, Baker DW. Comparative effectiveness of multifaceted outreach to initiate colorectal cancer screening in community health centers: a randomized controlled trial. J Gen Intern Med. 2015;30(8):1178–84.
48.
Greiner KA, Daley CM, Epp A, James A, Yeh HW, Geana M, Born W, Engelman KK, Shellhorn J, Hester CM, et al. Implementation intentions and colorectal screening: a randomized trial in safety-net clinics. Am J Prev Med. 2014;47(6):703–14.CrossRefPubMedPubMedCentral
49.
Gupta S, Halm EA, Rockey DC, Hammons M, Koch M, Carter E, Valdez L, Tong L, Ahn C, Kashner M, et al. Comparative effectiveness of fecal immunochemical test outreach, colonoscopy outreach, and usual care for boosting colorectal cancer screening among the underserved: a randomized clinical trial. JAMA Intern Med. 2013;173(18):1725–32.PubMedPubMedCentral
50.
Hendren S, Winters P, Humiston S, Idris A, Li SX, Ford P, Specht R, Marcus S, Mendoza M, Fiscella K. Randomized, controlled trial of a multimodal intervention to improve cancer screening rates in a safety-net primary care practice. J Gen Intern Med. 2014;29(1):41–9.CrossRefPubMed
51.
Jandorf L, Gutierrez Y, Lopez J, Christie J, Itzkowitz SH. Use of a patient navigator to increase colorectal cancer screening in an urban neighborhood health clinic. J Urban Health. 2005;82(2):216–24.CrossRefPubMedPubMedCentral
52.
Jean-Jacques M, Kaleba EO, Gatta JL, Gracia G, Ryan ER, Choucair BN. Program to improve colorectal cancer screening in a low-income, racially diverse population: a randomized controlled trial. Ann Fam Med. 2012;10(5):412–7.CrossRefPubMedPubMedCentral
53.
Lasser KE, Murillo J, Lisboa S, Casimir AN, Valley-Shah L, Emmons KM, Fletcher RH, Ayanian JZ. Colorectal cancer screening among ethnically diverse, low-income patients: a randomized controlled trial. Arch Intern Med. 2011;171(10):906–12.CrossRefPubMed
54.
Levy BT, Daly JM, Xu Y, Ely JW. Mailed fecal immunochemical tests plus educational materials to improve colon cancer screening rates in Iowa research network (IRENE) practices. J Am Board Fam Med. 2012;25(1):73–82.CrossRefPubMed
55.
Levy BT, Xu Y, Daly JM, Ely JW. A randomized controlled trial to improve colon cancer screening in rural family medicine: an Iowa research network (IRENE) study. J Am Board Fam Med. 2013;26(5):486–97.CrossRefPubMed
56.
Potter MB, Walsh JM, Yu TM, Gildengorin G, Green LW, McPhee SJ. The effectiveness of the FLU-FOBT program in primary care a randomized trial. Am J Prev Med. 2011;41(1):9–16.CrossRefPubMed
57.
Roetzheim RG, Christman LK, Jacobsen PB, Cantor AB, Schroeder J, Abdulla R, Hunter S, Chirikos TN, Krischer JP. A randomized controlled trial to increase cancer screening among attendees of community health centers. Ann Fam Med. 2004;2(4):294–300.CrossRefPubMedPubMedCentral
58.
Singal AG, Gupta S, Tiro JA, Skinner CS, McCallister K, Sanders JM, Bishop WP, Agrawal D, Mayorga CA, Ahn C, et al. Outreach invitations for FIT and colonoscopy improve colorectal cancer screening rates: a randomized controlled trial in a safety-net health system. Cancer. 2016;122(3):456.CrossRefPubMed
59.
Thompson B, Coronado G, Chen L, Islas I. Celebremos la salud! A community randomized trial of cancer prevention (United States). Cancer Causes Control. 2006;17(5):733–46.CrossRefPubMed
60.
Tu SP, Taylor V, Yasui Y, Chun A, Yip MP, Acorda E, Li L, Bastani R. Promoting culturally appropriate colorectal cancer screening through a health educator: a randomized controlled trial. Cancer. 2006;107(5):959–66.CrossRefPubMed
61.
Coronado GD, Vollmer WM, Petrik A, Aguirre J, Kapka T, Devoe J, Puro J, Miers T, Lembach J, Turner A, et al. Strategies and opportunities to STOP colon cancer in priority populations: pragmatic pilot study design and outcomes. BMC Cancer. 2014;14:55.CrossRefPubMedPubMedCentral
62.
Sarfaty M, Feng S. Uptake of colorectal cancer screening in an uninsured population. Prev Med. 2005;41(3–4):703–6.CrossRefPubMed
63.
Larkey L. Las mujeres saludables: reaching Latinas for breast, cervical and colorectal cancer prevention and screening. J Community Health. 2006;31(1):69–77.CrossRefPubMed
64.
Tu SP, Chun A, Yasui Y, Kuniyuki A, Yip MP, Taylor V, Bastani R. Adaptation of an evidence-based intervention to promote colorectal cancer screening: a quasi-experimental study. Implement Sci. 2014;9:85.
65.
Wu TY, Kao JY, Hsieh HF, Tang YY, Chen J, Lee J, Oakley D. Effective colorectal cancer education for Asian Americans: a Michigan program. J Cancer Educ. 2010;25(2):146–52.CrossRefPubMed
66.
Potter MB, Yu TM, Gildengorin G, Yu AY, Chan K, McPhee SJ, Green LW, Walsh JM. Adaptation of the FLU-FOBT program for a primary care clinic serving a low-income Chinese American community: new evidence of effectiveness. J Health Care Poor Underserved. 2011;22(1):284–95.PubMed
67.
Redwood D, Holman L, Zandman-Zeman S, Hunt T, Besh L, Katinszky W. Collaboration to increase colorectal cancer screening among low-income uninsured patients. Prev Chronic Dis. 2011;8(3):A69.PubMedPubMedCentral
68.
Sarfaty M, Feng S. Choice of screening modality in a colorectal cancer education and screening program for the uninsured. J Cancer Educ. 2006;21(1):43–9.CrossRefPubMed
69.
Ritvo P, Myers RE, Serenity M, Gupta S, Inadomi JM, Green BB, Jerant A, Tinmouth J, Paszat L, Pirbaglou M, et al. Taxonomy for colorectal cancer screening promotion: lessons from recent randomized controlled trials. Prev Med. 2017;101:229-234.
70.
Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, Altman DG, Barbour V, Macdonald H, Johnston M, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014;348:g1687.CrossRefPubMed
71.
Harvey G, Kitson A. PARIHS revisited: from heuristic to integrated framework for the successful implementation of knowledge into practice. Implement Sci. 2016;11:33.CrossRefPubMedPubMedCentral
72.
Waltz TJ, Powell BJ, Matthieu MM, Damschroder LJ, Chinman MJ, Smith JL, Proctor EK, Kirchner JE. Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: results from the expert recommendations for implementing change (ERIC) study. Implement Sci. 2015;10:109.CrossRefPubMedPubMedCentral
73.
Almirall D, Nahum-Shani I, Sherwood NE, Murphy SA. Introduction to SMART designs for the development of adaptive interventions: with application to weight loss research. Transl Behav Med. 2014;4(3):260–74.CrossRefPubMedPubMedCentral
74.
National Cancer Institute. Upcoming Cancer Moonshot℠ Funding Opportunities. Accelerating colorectal cancer screening and follow-up through implementation science (ACCSIS). https://​www.​cancer.​gov/​research/​key-initiatives/​moonshot-cancerinitiative​/​funding/​upcoming#accsis. Accessed 27 Nov 2017.
75.
Centers for Disease Control and Prevention. Colorectal Cancer Control Program (CRCCP). August 2017. https://​www.​cdc.​gov/​cancer/​crccp/​index.​htm. Accessed 27 Nov 2017.
76.
Davis MM, Keller S, DeVoe JE, Cohen DJ. Characteristics and lessons learned from practice-based research networks (PBRNs) in the United States. J Healthc Leadersh. 2012;4:107–16.PubMedPubMedCentral
77.
Marshall M, Pagel C, French C, Utley M, Allwood D, Fulop N, Pope C, Banks V, Goldmann A. Moving improvement research closer to practice: the researcher-in-residence model. BMJ Qual Saf. 2014;23(10):801–5.CrossRefPubMedPubMedCentral
78.
Wheeler SB, Davis MM. “Taking the bull by the horns”: four principles to align public health, primary care, and community efforts to improve rural cancer control. J Rural Health. 2017;33(4):345–9.CrossRefPubMed
79.
Pham R, Cross S, Fernandez B, Corson K, Dillon K, Yackley C, Davis MM. “Finding the right FIT”: rural patient preferences for fecal immunochemical test (FIT) characteristics. J Am Board Fam Med. 2017;30(5):632–44.CrossRefPubMed
80.
Glasgow RE, Chambers D. Developing robust, sustainable, implementation systems using rigorous, rapid and relevant science. Clin Transl Sci. 2012;5(1):48–55.CrossRefPubMedPubMedCentral
81.
Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review--a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005;10(Suppl 1):21–34.CrossRefPubMed
82.
Rycroft-Malone J, McCormack B, Hutchinson AM, DeCorby K, Bucknall TK, Kent B, Schultz A, Snelgrove-Clarke E, Stetler CB, Titler M, et al. Realist synthesis: illustrating the method for implementation research. Implement Sci. 2012;7:33.CrossRefPubMedPubMedCentral
83.
Pawson R. Evidence-based policy: a realist perspective. Thousand Oaks, CA: Sage; 2006.CrossRef
84.
Anderson R. New MRC guidance on evaluating complex interventions. BMJ. 2008;337:a1937.CrossRefPubMed
85.
National Institutes of Health. Research & Training: Rigor and Reproducibility. https://​www.​nih.​gov/​researchtraining​/​rigor-reproducibility. Accessed 27 Nov 2017.
86.
Hawe P, Shiell A, Riley T, Gold L. Methods for exploring implementation variation and local context within a cluster randomised community intervention trial. J Epidemiol Community Health. 2004;58(9):788–93.CrossRefPubMedPubMedCentral