Evidence Synthesis Methodologies in the Health Sciences

Full text screening yields a final set of included studies. Before extracting data, however, it's essential to assess risk of bias in those studies. This may also be referred to as quality assessment or critical appraisal.

According to Finding What Works in Health Care: Standards for Systematic Reviews, published by the Institute of Medicine:

"If an SR is to be based on the best available evidence on the comparative effectiveness of interventions, it should include a systematic, critical assessment of the individual eligible studies. The SR should assess the strengths and limitations of the evidence so that decision makers can judge whether the data and results of the included studies are valid."

As with screening, two independent reviewers should critically appraise each study. Risk of bias assessment is required for systematic reviews but not necessarily for other forms of evidence synthesis. Review the methodological guidelines you're following for a more thorough understanding of this step.

Retracted studies should not be relied upon in evidence synthesis. Some strategies for determining if your included studies may have been retracted:

Look for terms in abstracts or other reference metadata such as: retraction, retracted, erratum, errata, "expression of concern", comments, author responses to comments, or "update in: [article title]".

Search for retraction information in databases like PubMed ("retracted article") or Embase ("tombstone").

Check your citation management tool, such as Zotero or EndNote, both of which flag retracted articles based on Retraction Watch data.

The browser extensions LibKey Nomad and PubPeer also visibly flag items as retracted.

Consult the Retraction Watch dataset.

Quality assessment tools are designed based on study type. If your evidence synthesis project includes multiple study designs, you may need more than one tool to assess risk of bias.