Skip to Main Content

Systematic Reviews

An introduction to systematic review methodology

Upcoming Events: Systematic Reviews

Interested in learning more about systematic reviews in medicine and the health sciences? Belmont's Medical Library and Bunch Library are delighted to offer our three-workshop series in January-February 2025, which will provide an introduction to systematic review processes and how librarians can help. Explore the workshops below, and make sure to register to receive the workshop link:
What is a Systematic Review? 

Date: 24 January 2025, 12:00 PM, on Teams (Register here)

Description: While systematic reviews are generally considered to be at the top of the ‘evidence pyramid,’ not all are created equal! Come and learn about the purpose and steps of systematic reviews, the many types to choose from, and standards and guidelines available.

Which Review is Right for You?

Date: 31 January 2025, 12:00 PM, on Teams (Register here)

Description: Systematic, scoping, rapid, integrative… how do I know what review type is right for my research question? Come and discover the possibilities and pitfalls of different review types, and learn how to identify what types might be best for your research.

Systematic Approaches to Literature Reviews

Date: 7 February 2025, 12:00 PM, on Teams (Register here)

Description: There is a proliferation of reviews and review types, some with more methodological guidance than others. While systematic reviews are viewed as the gold standard, narrative reviews can contribute valuable new knowledge to a field. Come and learn how to take a systematic approach to any review to increase rigor and your chances of publication!  

What is a Systematic Review?

A systematic review is a ‘study about studies’ which focuses on a specific research question that it must try to answer by using explicit, transparent and reproducible methods to identify, critically appraise and synthesize results of similar studies to generate evidence-based findings. (From Stephanie Roth’s Introduction to Systematic Reviews Course. June 2020).  
Identify: Review teams develop search strategies for multiple research databases and resources, and screen their findings against pre-specified eligibility criteria. 
Appraise: Once all the evidence has been located and screened, review teams evaluate the quality of the evidence, assessing to understand where more research may be needed. 
Synthesize: Finally, the review team brings together the different pieces of evidence, identifying trends, issues, or patterns in the data. The rigorous systematic review methodology makes these reviews a great tool for clinicians, educators, and stakeholders involved in decision-making or best practice recommendations. 

Key Characteristics of a Systematic Review

  • Time-intensive: A typical systematic review can take 12-18 months to complete. 
  • Collaboration-intensive: The systematic review’s methodology requires multiple researchers to screen and evaluate evidence. A team of at least three is required. 
  • Protocol-based: Before beginning a systematic review, the team will pre-define their research question, expected outcomes, search strategy, and inclusion/exclusion criteria in a protocol. This helps mitigate potential bias or data manipulation.  
  • Research-question based: Systematic reviews are rooted in clear, focused questions. Many reviews will use the PICO framework or something similar to build their question. 
  • Exhaustive: The goal of a systematic review is to locate all of the available evidence for your question. 

Systematic Reviews Are Not...

  • Exclusive to medicine or healthcare: Systematic reviews are commissioned or developed for many different disciplines interested in making evidence-informed decisions. 
  • Exclusive to quantitative research: While many systematic reviews analyze studies that created quantitative data, qualitative and mixed methods reviews do exist and follow the systematic review methodology. 
  • The only “good” review type: Different types of reviews are helpful depending on your context and question. Meet with a librarian to discuss if a systematic review approach is right for your team. 
  • Exhausting (hopefully!): While systematic reviews can be time-intensive, the guiding methodology and opportunity for collaboration is invigorating to many researchers at all career levels. 

Systematic Reviews vs. Narrative/Literature Reviews

  Narrative Review  Systematic Review 
Purpose 
  • Summarizes and critiques the body of literature on a specific issue or topic 
  • Provides context for research, identifies gaps in the literature, and situates research within an area of study 
  • Often included with introduction to research study 
  • Can be publication in its own right, and provides a deeper look at a specific issue or topic 
  • Seeks to collate evidence that fits pre-specified eligibility criteria in order to answer a specific research question 
  • A stand-alone publication 
  • A rigorous research methodology  
Review Question  More general and broader  Begins with a focused, answerable clinical question (PICO framework) 
Authorship  Can be one or more authors  Needs a team with some SR methodology expertise – at least 3 people 
Searching for Studies  Searches may be ad hoc and based on what the author is already familiar with. Searches are not exhaustive or fully comprehensive.  Uses comprehensive and exhaustive literature searches in multiple sources.
Study Selection  Often lack clear reasons for why studies were included or excluded from the review.  Establishes pre-specified, explicit, and reproducible eligibility criteria.
Assessing the Quality of Included Studies  Often do not consider study quality or potential biases in study design.  Employs comprehensive analysis of quality and validity of findings 
Synthesis of Existing Research  Conclusions are more qualitative and may not be based on study quality.  Bases conclusion on quality of the studies and provide recommendations for practice or to address knowledge gaps. 
Timeline Can be completed in a few weeks to months.  Typically takes up to 1 year or more to complete.

*For more information how to conduct a narrative literature review see: https://medlib.belmont.edu/LitReviews  

Standards and Guidelines

One of the nice things about systematic reviews is that there are well-established standards and guidelines for this research methodology.  

JBI Manual for Evidence Synthesis  

  • guides authors who wish to conduct systematic and scoping reviews following JBI methodologies 
  • Each chapter is devoted to the synthesis of different types of evidence to address different types of clinical and policy-related questions. 
  • Easy to apply, step-by-step guidance 

Cochrane Handbook for Systematic Reviews of Interventions 

  • The official guide that describes preparing and maintaining Cochrane systematic reviews 
  • includes guidance on the standard methods applicable to every review (planning a review, searching and selecting studies, data collection, risk of bias assessment, statistical analysis, GRADE and interpreting results) 

Reporting Guidelines 

Appraising a Systematic Review

While systematic reviews are generally considered to be at the top of the “pyramid of evidence,” not all systematic reviews are created equal. Like any other research, they should be discussed, critiqued, and updated. Here are some tools for appraising systematic reviews: 

Critical Appraisal Skills Programme Systematic Review Checklists (CASP) 

  • Use these checklists to analyze how a systematic review was conducted. 

Risk of Bias Assessment Tool for Systematic Reviews (ROBIS) 

  • Use this tool to (1) assess relevance (optional), (2) identify concerns with the review process and (3) judge risk of bias for systematic reviews. 

Systematic Review Examples and Further Reading