Skip to Main Content

Evidence Synthesis Methodologies in the Health Sciences

Principles of Systematic Searching

Evidence synthesis requires a systematic approach to searching. While conducting any literature search is a thoughtful and detailed process, systematic searching is unique.

A systematic search is:

  • Comprehensive or exhaustive: it should should be as thorough as possible to avoid missing any relevant information.
  • Transparent: it should be clear what choices you made in designing your search strategy and why you made them.
  • Well-documented and reproducible: as with the methods for any research study, readers should be able to replicate your work. That means all your search strategies should be provided in full.

Searches performed for evidence syntheses are sometimes described as requiring high sensitivity or recall. A highly sensitive search is inevitably going to be lower in precision than a less sensitive one; that is, it will contain more irrelevant results than would a less comprehensive search.

A well-designed and executed systematic search is essential to any high quality evidence synthesis project.

Free Text vs. Controlled Vocabulary

There are two major categories of language used in systematic searching, both of which are essential to an effective search.

  • Free text terms are any words you can think of related to your topic. They may include scholarly or technical language, terms unique to certain disciplines, modern or outdated terminology, or different forms of the same words (singular, plural, regional spelling differences). Free text is used to search information provided by researchers, such as titles, abstracts, and author-supplied keywords. 
  • Controlled vocabulary, sometimes referred to as subject headings, descriptors, or index terms, are the standardized terminology used by databases to label references based on their content. Controlled vocabulary are generally arranged in a hierarchical thesaurus that explains what the terms mean and how they are related. Many, but not all, scholarly databases have a unique controlled vocabulary. Two examples are PubMed, which uses MeSH (Medical Subject Headings), and APA PsycInfo, which uses the APA Thesaurus of Psychological Index Terms.

Because exhaustive searching means considering all the ways relevant sources may be described, systematic searches must include both free text and controlled vocabulary.

Learn more about identifying useful free text terms and controlled vocabulary on the Plan Your Search page of this guide.

Controlled vocabulary varies from database to database. The same article may appear in PubMed, APA PsycInfo, and Embase, but be described very differently. For example, take the following article:

Attanasio, L. B., & Hardeman, R. R. (2019). Declined care and discrimination during the childbirth hospitalization. Social Science & Medicine, 232, 270–277. https://doi.org/10.1016/j.socscimed.2019.05.008

In PubMed, this is assigned several Medical Subject Headings from PubMed's controlled vocabulary, abbreviated as MeSH. This is an excerpted list: Attitude of Health Personnel; Black or African American; Delivery, Obstetric; Hospitalization; Patient Acceptance of Health Care; Pregnancy; Professional-Patient Relations; Racism

In APA PsycInfo, which uses the APA Thesaurus of Psychological Index Terms, the subject headings are: Birth; Discrimination; Hospitalization; Therapeutic Processes; Black People

In Embase, which uses the Emtree thesaurus, some of the assigned index terms include: childbirth; ethnicity; health care personnel; hospitalization; infant; punishment; race; stereotypy [sic]

You can see that, despite some overlap in how these three databases describe the same article, there are lots of unique terms. The hierarchies from which the terms are chosen will also be different. You can use controlled vocabulary in a database to search its contents, but you can also use terms from one database as free text in others. Looking at controlled vocabulary across several databases when developing your search can help expand and enhance your free text terms. 

Boolean Logic: AND, OR, and NOT

Boolean operators connect your ideas in a logical way that databases understand. The most common operators are AND, OR, and NOT.

AND is used to combine main ideas. For example, a search for mobile health applications AND cardiovascular exercise returns a set of citations that mentions both topics (the green shaded area in the center):

Boolean AND Venn diagram

OR is used to look for citations that match any one of a set of related terms. For example, a search for fitness apps OR health apps will find citations mentioning any of those phrases (the entire blue shaded area).

Boolean OR Venn diagram

NOT is used to remove results. For example, cardiovascular exercise NOT swimming returns a set of citations matching the phrase cardiovascular exercise but without the term swimming (the blue area only, excluding all swimming citations).

Boolean NOT Venn diagram

Using NOT is risky: you may inadvertently exclude something from your search results that would have been relevant. In the same example, imagine an article with an abstract that states: "All types of cardiovascular exercise except for swimming were studied." In that case, the article is not about swimming, so it would be relevant, but because the word swimming appeared in the abstract, the citation would be excluded from the search results. For this reason, NOT is generally not recommended in evidence synthesis searching.

Examples: Traditional vs. Systematic Searches

A basic (non-systematic) search in PubMed for concepts represented in the Boolean examples above might be:

mobile apps AND cardiovascular exercise

A more complex (but still not systematic) PubMed search might be:

(mobile apps OR health apps) AND (cardiovascular exercise OR running)

These might be good exploratory searches (see the Plan Your Search page of this guide), but they do not meet the standards of evidence synthesis searches.

A non-exhaustive, but far more systematic, search in PubMed for this topic is below, duplicated from an actual PubMed search history. You can learn more about searching PubMed on the Build Your Core Search Strategy page of this guide.

screen capture of systematic PubMed search