Systematic reviews attempt to collect all existing evidence on a specific topic to answer a specific research question. To do one well takes a lot of time and effort, and careful attention to decisions before fully diving in can save you time in the long run.
Defining your research question takes time and thoughtful conversation with mentors, collaborators, or clients. What will the review be used for? Is there a specific decisional dilemma you are attempting to address? What interventions, in what populations, in what settings are of most interest? Be as specific and narrow as possible - this will help you identify the most relevant studies. At this stage, start thinking about the review PICO(TS) you will have to specify later: the population, intervention, outcome(s), comparators, timing, and settings of interest for your review.
Familiarize yourself with other systematic reviews in your topic area and prepare a precise statement of what gap yours will fill (PubMed, The Cochrane Library, and Campbell Collaboration are good places to start). Are you updating the evidence base because you know of new evidence? Are you exploring an aspect of the topic that was excluded from previous reviews? Is it truly a new review? Establishing reputation by authoring one’s own systematic review is not a sufficient reason for proceeding—in the past decade, there has been at least a 3-fold increase in the number of published systematic reviews, with the quality and reporting varying widely.1 Consider how you might use or update existing systematic reviews rather than conducting a de novo review. In many cases, a rapid, scoping, narrative, or other non-systematic review may be a more appropriate and less time-consuming method.
Akin to a protocol for a randomized trial, a systematic review protocol articulates your predetermined methodologic decisions. At this stage, you will formulate your inclusion and exclusion criteria for selecting studies eligible for your review. What study designs will you allow? What outcomes will and will not be included? Will you require that those outcomes be reported at a certain follow-up time point? Your review methods must be transparent enough that an independent research group could replicate and find the same body of evidence and results. In addition to describing your research questions and eligibility criteria, your protocol should detail the information sources and search strategy, study screening and selection process, methods for data extraction and risk of bias assessment, and plans for data synthesis, including processes for qualitative or quantitative (meta-analysis, network meta-analysis) analyses. Consider publishing your protocol in a peer-reviewed journal (the journal Systematic Reviews routinely publishes systematic review protocols) and registering your review at https://www.crd.york.ac.uk/prospero/.
There are many good resources that walk through the standards and methods for conducting a reputable systematic review including the IOM Standards for Systematic Reviews, the Cochrane Handbook for Systematic Reviews of Interventions, and the Agency for Healthcare Research and Quality’s Methods Guide for Effectiveness and Comparative Effectiveness Reviews. Following these standards can help you produce a trustworthy and useful research product.