Systematic Review Protocol to Evaluate the Evidence Informing the Physical Activity Guidelines for Americans Midcourse Report: Implementation Strategies for Older Adults

Alison Vaux-Bjerke¹, Deborah H. John², and Katrina L. Piercy¹

¹ Office of Disease Prevention and Health Promotion (ODPHP), U.S. Department of Health and Human Services, Rockville, MD, U.S.A.

² Oregon State University, College of Public Health and Human Sciences, U.S.A.

Abstract

The Physical Activity Guidelines for Americans (Guidelines) advises older adults to be as active as possible. Yet, despite the well documented benefits of physical activity just 12.8% of those ages 65 and older meet the Guidelines. To address this, the U.S. Department of Health and Human Services (HHS) developed a Midcourse Report focused on effective strategies to improve older adult physical activity behaviors. The first step in this process was a systematic literature review. A literature review team was contracted to examine the evidence on key settings and effective behavioral intervention strategies, as well as effective policy, systems, and environmental (PSE) approaches, to improve physical activity among older adults. The PSE search employed an equity-centered framework adapted to researching PSE approaches for improving physical activity outcomes in older adults. Sixteen thousand eight hundred and eighty-three titles and abstracts were screened, and 734 full articles were reviewed for inclusion. Of those, 64 original research articles were included for the final review to answer two questions, one (plus 5 sub-questions) focused on Settings/Strategies literature (45 studies) and one (plus 2 sub-questions) focused on PSE literature (19 studies). The literature review process identified key settings and evidence-based strategies to support older adults in becoming more physically active, and provides a foundation for the Physical Activity Guidelines for Americans Midcourse Report: Implementation Strategies for Older Adults. More research is needed to address how factors related to equity and psychosocial constructs influence physical activity behaviors among older adults.

Keywords: Physical activity, policy, older adults, systematic review, Policy-Systems-Environment, aerobic physical activity, muscle strengthening physical activity

Introduction

The Physical Activity Guidelines for Americans (Guidelines) (U.S. Department of Health and Human Services (HHS), 2018) serves as the benchmark and primary, authoritative voice of the federal government for providing science-based guidance on physical activity, fitness, and health for Americans. The most recent edition, released in 2018, provides evidence-based recommendations for Americans ages 3 and older to safely get the physical activity they need to stay healthy (HHS, 2018). In 2013, five years after the release of the first edition of the Guidelines, HHS released a midcourse report (Physical Activity Guidelines for Americans Midcourse Report Subcommittee of the President’s Council on Fitness, Sports & Nutrition (PAG Midcourse Subcommittee), 2013). This report focused on strategies to increase physical activity among youth and focused on five key areas where youth live, learn, and play – preschool and childcare centers, schools, family and home, community (built environment), and primary care medical settings (PAG Midcourse, 2013). The next midcourse report focused on older adults (ages 65 and older).

The benefits of regular physical activity occur throughout life and are essential for healthy aging. Research suggests it is never too late to start being physically active. Despite the many benefits of physical activity, only 12.8% of adults over age 65 meet the aerobic and muscle-strengthening Guidelines (HHS Office of Disease Prevention and Health Promotion (ODPHP), 2022). This rate may be influenced by several factors, as barriers to physical activity differ from individual to individual and are influenced by socioeconomic, cultural, built environment, and other community factors.

The Guidelines contains quantitative recommendations for older adults but does not include implementation strategies. Therefore, a literature review was conducted to identify successful interventions to promote increased physical activity and adherence to the key guidelines for older adults and summarized in the Physical Activity and Older Adults Systematic Literature Review (ICF Next, 2023). The Physical Activity Guidelines for Americans Midcourse Report: Implementation Strategies for Older Adults (Midcourse Report) (HHS, 2023) serves to further the breadth of the Guidelines to facilitate the implementation of proven programs and other strategies that can increase levels of physical activity among older adults. This paper outlines the literature review methodology to support the Midcourse Report.

Methods: Literature Review

In 2022, HHS contracted with a Literature Review Team to review the evidence on effective strategies to increase physical activity among older adults. This work was supported by the President’s Council on Sports, Fitness & Nutrition (PCSFN) Science Board (Science Board), made up of 11 experts in physical activity and older adult populations.

The Literature Review Team used a methodology supported by best practices for systematic literature reviews developed by the United States Department of Agriculture’s (USDA) Nutrition Evidence Systematic Review (NESR) (USDA NESR Branch, 2023), the Agency for Healthcare Research and Quality (AHRQ) (AHRQ, 2014), the Cochrane Collaboration (Higgins et al., 2022), and the Health and Medicine Division of the National Academies of Sciences, Engineering, and Medicine standards to review, evaluate, and synthesize published, peer-reviewed physical activity research (Institute of Medicine (US) Committee on Standards for Systematic Reviews of Comparative Effectiveness Research, 2011). This review process was largely guided by the approach taken to review the literature for the 2018 Physical Activity Guidelines Advisory Committee Scientific Report(2018 Physical Activity Guidelines Advisory Committee, 2018; Torres et al., 2018). Paralleling the 2018 process, this protocol-driven review approach was undertaken to maximize transparency, minimize bias, and ensure the review conducted was timely, relevant, and high quality. There are two major distinctions between this review and that conducted by the 2018 Physical Activity Guidelines Advisory Committee: 1) the decision to review original articles instead of using a “review of reviews” approach; and 2) to focus on research with physical activity outcomes as opposed to health outcomes.

All work completed by the Literature Review Team was under the direction and review of ODPHP, on behalf of the National Institutes of Health (NIH), the Centers for Disease Control and Prevention, and the PCSFN. The Literature Review Team coordinated the literature review process, developed an abstraction tool and accompanying abstraction and triage guides, and implemented training and quality control protocols. Several groups supported the literature review work:

• Librarians reviewed search strategies and provided guidance as needed.

• The Triage Team conducted title and abstract triage of articles identified through the literature searches.

• The Abstraction Team engaged in rigorous training before abstracting data from included articles. A portion of this group also assessed risk of bias on a subset of the included articles.

• The Science Board identified, aggregated, organized, and analyzed the scientific literature.

A six-step process was used to examine the literature:

• Step 1: Develop systematic review questions

• Step 2: Develop systematic review strategy

• Step 3: Search, screen, and select evidence to review for each question

• Step 4: Abstract data and assess the risk of bias of the research

• Step 5: Describe the evidence

• Step 6: Complete evidence portfolios and draft Scientific Report

Step 1: Developing Systematic Review Questions

Following the cadence of previous editions of the Physical Activity Guidelines for Americans with an interim Midcourse Report, HHS initiated the process to review the scientific literature focused on effective approaches to promote physical activity among older adults. ODPHP outlined a need to examine intervention strategies and key settings that are effective in promoting movement and achievement of the key guidelines for physical activity for older adults. Additional factors of interest for the literature review included how engagement in physical activity interventions may influence mental health, well-being, social connection, and other related social and emotional factors; as well as how interventions implemented as policy, systems, and environmental approaches to change the context influence physical activity in older adult populations.

Solidifying Systematic Review Questions

The Literature Review Team developed research questions focused on the previously specified topics. The research questions and corresponding sub-questions are as follows:

Question 1: What are effective intervention strategies to increase physical activity among older adults?

a) Does the mode of delivery (e.g., virtual, in person, phone) impact the effectiveness of interventions?

b) Does the setting impact the effectiveness of the interventions?

c) What barriers exist to engaging or participating in the intervention? What are the retention, attrition, and/or attendance rates?

d) Do personal characteristics (e.g., ability, age, sex, race/ethnicity, socioeconomic status) or chronic health conditions influence participation?

e) Do interventions assess changes in participant mental health, quality of life, well-being, resilience, or social connection and isolation?

Question 2: What are effective policy, systems, and environmental (PSE) strategies to increase physical activity among older adults?

a) Is there a dose-response relation between the scope and reach of the PSE strategy and “success”?

b) Does the “success” of the PSE strategy vary by geographical location or by sociodemographic subgroup?

Step 2: Develop Systematic Review Strategy

Develop Analytical Frameworks

Analytical frameworks were developed for each research question. Analytical frameworks are graphic representations used to lay the groundwork and initial parameters for each search. The frameworks served as a guide to define key variables, inform the inclusion and exclusion criteria, and develop the literature review strategy. These frameworks were created using the PICO method (population, intervention, comparison, and outcomes) (Higgins et al., 2022) and were modeled on the approach used for the 2018 Physical Activity Guidelines Advisory Committee Scientific Report (2018 Physical Activity Guidelines Advisory Committee, 2018; Torres et al., 2018). The frameworks were constructed during weekly meetings (see supplementary materials).

Develop Inclusion and Exclusion Criteria

The Literature Review Team created inclusion and exclusion criteria for each research question. The template used to draft inclusion and exclusion criteria was modeled off the approach used for the 2018 Physical Activity Guidelines Advisory Committee Scientific Report (2018 Physical Activity Guidelines Advisory Committee, 2018), and adapted to meet the needs of this particular review. These templates were used to determine whether studies were eligible to be selected for each respective systematic literature review and whether studies would provide data to support the focal research questions. To promote consistency and relevance, and to account for scope parameters, each template included similar sections (Tables 1 and 2).

Table 1. Inclusion and Exclusion Criteria for Question 1: What are effective intervention strategies to increase physical activity among older adults?

Include

Published in English Language

English Language Publication

Peer-Reviewed Literature

Published From 2012 to 2022

Original Research

Human Participants

Intervention Study (Comparison Required)

Must Measure Physical Activity Outcome

Older Adults (minimum or mean age of 65 years or older)

Designs Include Randomized Controlled Trials, Non-Randomized Controlled Trials, and Quasi-Experimental Studies

Exclude

Studies of Older Adults in Long-Term, Memory, or Hospice Facilities

Studies of Disease-Specific Therapeutic Exercise Delivered in Health/Medical Facility

Table 2. Inclusion and Exclusion Criteria for Question 2: What are effective policy, systems, and environmental (PSE) strategies to increase physical activity among older adults?

Include

Published in English Language

Peer-Reviewed Literature

Published from 2012-2022

Original Research

Human Participants

Study Conducted in the United States

Intervention Study (PSE Intervention)

Must Measure Physical Activity Outcome

Older Adults, Middle Age (50+ years) and Older

Designs Include Non-Randomized Controlled Trials, Prospective Cohort, Retrospective Cohort, Case-Control, Cross-Sectional, Before and After, Geospatial, Environmental, and Surveillance

Exclude

Studies of Older Adults in Long-Term, Memory, or Hospice Facilities

Studies of Disease-Specific Therapeutic Exercise Delivered in Health/Medical Facility

Develop Search Strategy

A search strategy was created to identify peer-reviewed original research for each systematic review conducted. Each search strategy included search terms, Boolean logic to join terms, databases used, and key limits relevant to the inclusion criteria (e.g., research type, date of publication, language, study population, and filters specific to databases). The three databases included in each review were PubMed, CINHAL, and PsycINFO. These databases were selected due to the subject matter of articles included within each database.

The Literature Review Team developed sets of search terms most relevant to each review. These sets included terms capturing a broad range of articles based on older adult population, intervention study design, comparison approach, and physical activity outcomes. Once these search terms were drafted, library representatives from the literature review contractor and the NIH Library reviewed the search strategies and provided suggestions for updates. Throughout the search strategy development process, draft searches were run to assess the number of articles included in the search and if the collected articles represented the nature of the research questions. In response to these outputs, modifications were made to the search strategies as needed. The final search strategies were shared with the Science Board for review and affirmation.

Step 3: Search, Screen, and Select Evidence to Review

The searching and screening process was completed to collect a thorough body of original research needed to support each systematic review. Specifically, a primary search was completed within the original literature, and a supplemental search of existing meta-analyses and systematic literature reviews was conducted to support this process. A review of the original research was completed using the previously developed search strategies. Once results were generated, duplicates were removed, and results were triaged based on title and abstract.

Triage Training

Once the search strategy was implemented, each title/abstract underwent two rounds of review by members of the triage team. Members of the triage team were provided with thorough training and required to complete a certification process to ensure consistency between reviews prior to initiating triage. This training involved a comprehensive instructional presentation that was supported by a triage training manual that included detailed instructions, definitions, reporting instructions, response options, and example titles/abstracts. In addition to the formal training, members of the leadership team met one-on-one with triagers on an as-needed basis to promote consistency and accuracy. Prior to initiating the triage process, all potential triagers were required to complete a certification process on a subset of abstracts. Triagers who did not display a high level of consistency with the group on the practice assignment were not authorized to participate in the formal abstraction process.

Title and Abstract Triage

Triagers were instructed to first review titles to assess eligibility and then move to abstracts if the article appeared to be potentially relevant. Triagers were instructed to then include or exclude articles based on information provided within the abstract. If articles were removed from consideration at the abstract review stage, triagers were required to provide a reason. These reasons differed based on the search, but often included reasons such as ineligible age of participants, no physical activity outcome included, and/or no physical activity component of intervention reported. When conflicts existed in decisions made by screeners, discrepancies were resolved by a member of the triage team. The lists of included and excluded articles were shared with members of the Science Board.

Additional Supplemental Search Activities

Additional search activities were undertaken to further fortify the pool of articles collected through the initial search of the original literature. First, using the pre-established search terms, a systematic search was undertaken to identify relevant meta-analyses and systematic literature reviews that could potentially include original research articles relevant to the focal inclusion criteria. In tandem with this, a snowball approach (Wohlin, 2014) was also used to locate any additional meta-analyses and systematic literature reviews that could be deemed relevant. Reference pages and results tables from each of these meta-analyses and systematic literature reviews were hand-reviewed to identify original research articles that should be included within the review. These selected articles were reviewed in full-text and added to the pool if they fit the specified inclusion criteria.

Science Board members were encouraged to share additional articles that fit inclusion criteria with the Literature Review Team. These articles were identified through the Science Board’s expertise and familiarity with the subject matter. If an article was identified that met the inclusion criteria, it was reviewed in full text and abstracted by the Abstraction Team.

Full Text Review

Full text review for the list of included articles was conducted by members of the Literature Review Team. Two reviewers assessed each full-length article based on the inclusion criteria to determine whether it should be included or excluded from the final pool for review. Further, any articles that were identified as potentially ineligible during the abstraction process were added to the list of excluded articles.

Step 4: Abstract Data and Assess Risk of Bias

The abstraction process was used to collect and summarize key characteristics of each study that supported the systematic literature review purpose. The goal of abstraction was to (a) document key elements of each study for ease of review, and (b) use this information to present trends across the full body of evidence.

Abstraction Training

Abstractors were onboarded, trained, and certified to complete all abstraction activities. Abstraction candidates participated in a thorough and rigorous multi-phased process prior to initiating abstraction. This training involved a multi-hour instructional session. This session was supported by an abstractor training manual that included detailed instructions, definitions, reporting instructions, response options, example abstraction questions, and thoroughly annotated version of articles used in the training. In addition to the formal training, members of the Literature Review Team met with and/or delivered written feedback to the abstractors to ensure consistency and promote recalibration when needed. Prior to initiating the abstraction process, all potential abstractors were required to complete abstraction on practice articles. Abstractors who did not display a high level of consistency with the group during the practice sessions did not participate in the formal abstraction process.

Abstraction Process Explanation and Quality Control

Abstractors worked in pairs to independently review articles, abstract articles, and document findings. Abstractors were provided with random assignments of articles from members of the Literature Review Team. When discrepancies in abstraction were identified by the Literature Review Team, abstractors were asked to review and discuss these discrepancies. When discrepancies could not be settled among abstractors, members of the Literature Review Team reviewed the situation/materials and provided input and clarification to settle on a decision. The Literature Review Team conducted quality control and independently conducted a third round of abstraction for 20% of all articles included in each respective review. This quality control process was completed to ensure accuracy, clarity, and consistency in abstraction.

Data Documentation

An online database was created, and abstractors entered their data into this system using forms created by the Literature Review Team. All pairings of abstractors independently read and reviewed articles, abstracted key information, and entered it into the online database. After discrepancy resolution and quality control procedures were completed, the abstracted data was edited as needed and used to populate article evidence summary tables and inform trend tables demonstrating overarching themes in the data.

Assessing Risk of Bias

Articles were assessed for internal validity, using either the ROBINS-I (Sterne et al., 2016) or ROBINS-E tool (ROBINS-E Development Group, 2022). These tools assess risk of bias in studies that compare the health effects of exposures or interventions across a range of study types (e.g., RCT’s, observational, etc.). These tools are tailored by study design and pose different sets of questions based on whether a study is a randomized controlled trial (RCT), non-randomized controlled trial, or an observational study. The risk of bias assessment for each study was completed by two reviewers (from either the Abstraction Team or the Science Board). When discrepancies arose, the reviewers discussed and resolved discrepancies.

Additionally, the Policy, Systems, and Environments review used an equity-centered framework relevant to the research evidence and adapted to researching policy, systems, and environmental approaches for improving physical activity outcomes in older adults (Venkateswaran et al., 2023). The diversity-equity-inclusion frame was applied to studies assessed for risk of bias using ROBINS-E across relevant domains of bias (ROBINS-E Development Group, 2022) (i.e., confounding, selection of participants into the study, classification of exposures, departures from intended exposures, missing data, measurement of outcomes, and selection of reported results, aligned with PICO model).

Step 5: Describe the Evidence

Evidence Portfolios

To facilitate the analysis of the evidence, the Literature Review Team prepared evidence portfolios for each question (see supplementary materials). The evidence portfolios documented the full process followed for both reviews, including the sources of evidence, conclusions, evidence grades, description of evidence, populations analyzed, individual evidence summary tables, risk of bias and quality assessment charts, search strategies, literature trees, references, and rationales for exclusion of articles during full-text triage.

Step 6: Complete Evidence Portfolios and Draft Scientific Report

Science Board members reviewed and deliberated on the body of evidence to develop conclusion statements that supported each of the research questions. Conclusion statements were tightly associated with the evidence, focused on general agreement among the studies around the independent variables and outcomes, and acknowledged areas of disagreement or limitations, where they existed. The conclusion statements reflected only the evidence reviewed and not information Science Board members might have known from another source.

Along with the evidence portfolios, a rubric was developed to guide the assessment and grading of the strength of the evidence supporting each conclusion statement (Table 3). The rubric was adapted from the 2018 Physical Activity Guidelines Advisory Committee (2018 Physical Activity Guidelines Advisory Committee, 2018) and the 2020 Dietary Guidelines Advisory Committee (2020 Dietary Guidelines Advisory Committee, 2020). Grading the strength of the evidence was based on applicability of the populations, exposures, and outcomes studied; generalizability to the population of interest; risk of bias and study limitations; quantity and consistency of findings across studies; and magnitude and precision of effect.

Discussion

The Physical Activity and Older Adults Systematic Literature Review (ICF Next, 2023) evaluated the current scientific literature on strategies to increase physical activity among older adults to inform the Physical Activity Guidelines for Americans Midcourse Report: Implementation Strategies for Older Adults (HHS, 2023). Several strategies emerged across a variety of settings; the most commonly researched were home, health care, and community. Several limitations to the Physical Activity and Older Adults Systematic Literature Review (ICF Next, 2023) and opportunities to strengthen the research base on physical activity interventions for older adults should be noted.

While the literature review looked at original research articles rather than systematic reviews or meta-analyses, most included studies did not measure, analyze, or disaggregate findings based on important individual or group characteristics. For the Settings and Strategies question, the team attempted to determine if the personal characteristics (e.g., ability, age, sex, race/ethnicity, socioeconomic status) influence physical activity participation, but there was insufficient evidence to yield any analysis. For the Policy, Systems, and Environments question, equity considerations were applied, particularly concepts of diversity, inclusion, and access to study samples and intervention contexts to determine inequalities in physical activity outcomes. The samples did not reflect diverse populations and therefore resulted in an inability to grade the evidence or provide specific analysis for different subsets of older adults. For example, of the 19 included studies, only three studies examined specific racial/ethnic minority populations; two studies reported findings relative to disability-mobility limitations; and one study reported findings relative to health (multiple sclerosis status). Only two studies compared exposure to urban versus rural community geographies. Most studies of policy, systems, and environmental interventions employed cross-sectional design, examining residential neighborhoods (exposure context) in relation to physical activity outcomes, typically self-reported.

Because of the importance of social connection and mental well-being, especially for older adults who are socially isolated or live alone, the review made efforts to examine these concepts in relation to physical activity outcomes and interventions. Unfortunately, most published studies of interventions for improving physical activity in older adults did not include social or mental well-being outcomes, such as social cohesion, quality of life, resilience, or mental health status.

Lastly, few studies employed longitudinal designs, assessed long-term maintenance of outcomes, or investigated strategies to improve retention or prevent drop out in interventions to increase physical activity among the diversity of older adults, so physical activity maintenance, population and subpopulation effect remain uncertain.

The Physical Activity and Older Adults Systematic Literature Review (ICF Next, 2023) provides a foundation suggesting what strategies work and in which settings to support physical activity in older adults. Several limitations to the literature review can be used as areas to strengthen future research to ensure more diverse populations are recruited, studied, analyzed, and outcomes documented in the scientific literature.

Conclusion

The Systematic Review (ICF Next, 2023) and Midcourse Report (HHS, 2023) are useful for physical activity researchers; policy makers; exercise and health professionals; clinicians; gerontologists; built environment professionals; local, state, territorial, and Tribal leaders; and others working with older adults. These reports are necessary as a guide to apply evidence-based strategies to support older adults to be more physically activity and to expand the future evidence base to translate Guidelines into practice.

Correspondence should be addressed to

Alison Vaux-Bjerke, MPH, MCHES

Office of Disease Prevention and Health Promotion (ODPHP)

1101 Wootton Parkway, Suite 420

Rockville, MD 20852

alison.vaux-bjerke@hhs.gov

240-453-8272

Alison Vaux-Bjerke: 0000-0001-8930-7639

Deborah H. John: 0000-0002-0172-5252

Katrina L. Piercy: 0009-0009-8883-1645

Acknowledgments

The authors are grateful to the 2022 President’s Council on Sports, Fitness & Nutrition Science Board for their work on the literature review: Barbara J. Nicklas (Science Board Chair), Susan W. Buchholz, David E. Conroy, Cheryl Der Ananian, Loretta DiPietro, Mark Fenton, Deborah H. John, Nicole R. Keith, David X. Marquez, Jacqueline Osborne, and Dori Rosenberg. Additionally, the authors would like to acknowledge Sarah Caban, Rachel Fisher, Noelle Harada, Carolyn Hinton, and Malorie Polster for their contributions to this project.

Conflict of interest statement:

We have no conflicts of interest to disclose.

Author Contributions

Conceptualization: A.V.B. and K.L.P.; Data Curation: D.H.J.; Formal Analysis: D.H.J.; Funding Acquisition: A.V.B. and K.L.P.; Investigation: D.H.J.; Methodology: A.V.B., D.H.J., and K.L.P.; Project Administration: A.V.B. and K.L.P.; Resources: A.V.B. and K.L.P.; Supervision: A.V.B. and K.L.P.; Validation: D.H.J.; Writing – Original Draft: A.V.B., D.H.J., and K.L.P.; Writing – Review & Editing: A.V.B. and K.L.P.

Creative Commons License:

This work is licensed under a Creative Commons Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0).

References

Agency for Healthcare Research and Quality. (2014). Methods Guide for Effectiveness and Comparative Effectiveness Reviews. Rockville, MD: Agency for Healthcare Research and Quality. AHRQ Publication No. 10(14)-EHC063-EF.

2018 Physical Activity Guidelines Advisory Committee. (2018). 2018 Physical Activity Guidelines Advisory Committee Scientific Report. Washington, DC: U.S. Department of Health and Human Services. https://health.gov/our-work/nutrition-physical-activity/physical-activity-guidelines/current-guidelines/scientific-report

Dietary Guidelines Advisory Committee. (2020). Scientific Report of the 2020 Dietary Guidelines Advisory Committee: Advisory Report to the Secretary of Agriculture and the Secretary of Health and Human Services. Washington, DC: U.S. Department of Agriculture, Agricultural Research Service. https://www.dietaryguidelines.gov/2020-advisory-committee-report

Higgins, J.P.T., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M.J., Welch, V.A. (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.3. Chapter 3, “Defining the criteria for including studies and how they will be grouped for the synthesis.” Cochrane, 2022. Available www.training.cochrane.org/handbook.

ICF Next. (2023). Physical Activity and Older Adults Systematic Literature Review. Washington, DC: U.S. Department of Health and Human Services, 2023. https://health.gov/sites/default/files/2023-06/PAG%20Midcourse%20Literature%20Review%20Summary%20Report_for%20508c_06272023.pdf

Institute of Medicine (US) Committee on Standards for Systematic Reviews of Comparative Effectiveness Research, Eden, J., Levit, L., Berg, A., & Morton, S. (Eds.). (2011). Finding What Works in Health Care: Standards for Systematic Reviews. National Academies Press (US).

Physical Activity Guidelines for Americans Midcourse Report Subcommittee of the President’s Council on Fitness, Sports & Nutrition. (2013). Physical Activity Guidelines for Americans Midcourse Report: Strategies to Increase Physical Activity Among Youth. Washington, DC: U.S. Department of Health and Human Services. https://health.gov/sites/default/files/2019-09/pag-mid-course-report-final.pdf

ROBINS-E Development Group (Higgins, J., Morgan, R., Rooney, A., Taylor, K., Thayer, K., Silva, R., Lemeris, C., Akl, A., Arroyave, W., Bateson, T., Berkman, N., Demers, P., Forastiere, F., Glenn, B., Hróbjartsson, A., Kirrane, E., LaKind, J., Luben, T., Lunn, R., McAleenan, A., McGuinness, L., Meerpohl, J., Mehta, S., Nachman, R., Obbagy, J., O'Connor, A., Radke, E., Savović, J., Schubauer-Berigan, M., Schwingl, P., Schunemann, H., Shea, B., Steenland, K., Stewart, T., Straif, K., Tilling, K., Verbeek, V., Vermeulen, R., Viswanathan, M., Zahm, S., Sterne, J.). (2022). Risk Of Bias In Non-randomized Studies - of Exposure (ROBINS-E). Available from: https://www.riskofbias.info/welcome/robins-e-tool.

Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., Carpenter, J. R., Chan, A.-W., Churchill, R., Deeks, J. J., Hróbjartsson, A., Kirkham, J., Jüni, P., Loke, Y. K., Pigott, T. D., . . . Higgins, J. P. (2016). ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ, 355, i4919. https://doi.org/10.1136/bmj.i4919

Torres, A., Tennant, B., Ribeiro-Lucas, I., Vaux-Bjerke, A., Piercy, K., & Bloodgood, B. (2018). Umbrella and Systematic Review Methodology to Support the 2018 Physical Activity Guidelines Advisory Committee. Journal of Physical Activity & Health, 15(11), 805–810. https://doi.org/10.1123/jpah.2018-0372

U.S. Department of Health and Human Services, Office of Disease Prevention and Health Promotion. (2022). Healthy people 2030 topics and objectives: physical activity. https://health.gov/healthypeople/objectives-and-data/browse-objectives/physical-activity

U.S. Department of Health and Human Services. (2018). Physical Activity Guidelines for Americans, 2nd edition. Washington, DC: U.S. Department of Health and Human Services. https://health.gov/sites/default/files/2019-09/Physical_Activity_Guidelines_2nd_edition.pdf

U.S. Department of Health and Human Services. (2023). Physical Activity Guidelines for Americans Midcourse Report: Implementation Strategies for Adults. Washington, DC: U.S. Department of Health and Human Services. https://health.gov/sites/default/files/2023-08/PAG_MidcourseReport_508c_08-10.pdf

U.S. Department of Agriculture (USDA) Nutrition Evidence Systematic Review Branch. (2023). USDA Nutrition Evidence Systematic Review: Methodology Manual. U.S. Department of Agriculture, Food and Nutrition Service, Center for Nutrition Policy and Promotion, Nutrition Evidence Systematic Review. Available at: https://nesr.usda.gov/methodology-overview.

Venkateswaran, N., Feldman, J., Hawkins, S., Lewis, M. A., Armstrong-Brown, J., Comfort, M., Lowe, A., and Pineda, D. (2023). Bringing an Equity-Centered Framework to Research: Transforming the Researcher, Research Content, and Practice of Research. RTI Press Publication No. OP-0085-2301. Research Triangle Park, NC: RTI Press. https://doi.org/10.3768/rtipress.2023.op.0085.2301

Wohlin, C. (2014). Guidelines for snowballing in systematic literature studies and a replication in software engineering Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering, London, England, United Kingdom. https://doi.org/10.1145/2601248.2601268

Grade	Definition
Strong	The conclusion statement is based on a strong body of evidence as assessed by risk of bias, consistency, directness, precision, and generalizability. The level of certainty in the conclusion is strong, such that if new evidence emerges, modifications to the conclusion are unlikely to be required.
Moderate	The conclusion statement is based on a moderate body of evidence as assessed by risk of bias, consistency, directness, precision, and generalizability. The level of certainty in the conclusion is moderate, such that if new evidence emerges, modifications to the conclusion may be required.
Limited	The conclusion statement is based on a limited body of evidence as assessed by risk of bias, consistency, directness, precision, and generalizability. The level of certainty in the conclusion is limited, such that if new evidence emerges, modifications to the conclusion are likely to be required.
Grade Not Assignable	A conclusion statement cannot be drawn due to either a lack of evidence or evidence that has severe limitations related to risk of bias, consistency, directness, precision, and/or generalizability.