Hierarchy of Evidence: Study Types and Clinical Appraisal Examples

The hierarchy of evidence helps you find the study designs most likely to answer a clinical question reliably. For treatment effects, the usual starting order is systematic reviews of randomized trials, individual randomized trials, comparative observational studies, then uncontrolled reports and mechanistic reasoning. The order changes with the question. Check each study’s methods and relevance.

The Oxford CEBM introduction describes its hierarchy as a search aid, with exceptions that require judgment. A design label cannot tell you whether an effect matters to the person in front of you. This guide explains the study types, compares two fictional studies, and appraises a published trial.

Sources checked: November 3, 2026.

Study types in a treatment evidence hierarchy

This table compares study designs for treatment-effect questions. A review inherits important strengths and weaknesses from the studies it includes. A meta-analysis combines estimates; the quality of those estimates still depends on the underlying comparisons.

The Cochrane review methods distinguish a systematic review from the optional statistical step of meta-analysis. Read the review’s eligibility rules, search date and synthesis methods before relying on its conclusion. Several reports from one trial also represent one underlying study, not several independent confirmations.

Observational evidence can fill gaps in trials, particularly for rare harms, long follow-up or populations omitted from trials. Cochrane’s guidance on non-randomized studies explains these uses and their methodological challenges. For example, if sicker patients preferentially receive a treatment, an unadjusted comparison may make that treatment look harmful because of the starting differences. A larger sample does not remove that problem.

How to identify an evidence level

Use the Oxford CEBM 2011 Levels of Evidence, version 2.1, to classify Report B from the fictional HE-01 example:

1. Clinical question: in adults discharged after medication changes, does pharmacist follow-up reduce unplanned urgent visits within 90 days compared with usual discharge care? This belongs in the framework’s treatment-benefits row.
2. Study design: Report B randomly assigns 240 adults to the two strategies. It is an individual randomized trial.
3. Applicable level: a randomized trial starts at Level 2 in that row of the Oxford 2011 table. This is a design-based starting point; Oxford allows levels to be adjusted after appraisal, including downward for imprecision, quality problems or a mismatch with the clinical question.
4. Limitations to record: the trial comes from one hospital, with relatively few events and a wide confidence interval. Serious harms, acceptability and staffing costs are not measured. Record the level alongside these limitations.

Keep the evidence level, certainty and recommendation as separate judgments. Certainty concerns the body of evidence for an outcome; a recommendation also requires weighing benefits, harms, alternatives, feasibility and patient priorities. Oxford’s introductory guidance explains how to use its levels as a search aid while making those judgments.

Match the design to the question

Write the question before choosing a hierarchy. “Does the test detect disease accurately?” differs from “Does using the test improve patient outcomes?” The first needs evidence about classification; the second needs evidence about the consequences of testing and subsequent care.

The 2011 Oxford CEBM table provides separate paths for different questions. Useful starting points include:

• Treatment benefit: randomized comparisons and their systematic reviews.
• Diagnostic accuracy: patients receiving the test and an appropriate reference standard, interpreted with suitable blinding.
• Prognosis: cohorts followed from a relevant, reasonably consistent point in the condition’s course.
• Local prevalence: a representative, current sample of the population of interest.
• Rare or delayed harms: relevant observational evidence may supply information unavailable from trials.

Questions about why people avoid a service or how they experience treatment need another approach. Qualitative research can explore those experiences. GRADE-CERQual assesses confidence in findings from qualitative evidence syntheses through methodological limitations, coherence, adequacy and relevance. An interview study about barriers to attendance should be judged against that purpose.

For a treatment search, write out the PICO question and add the follow-up period: “Adults leaving hospital after medication changes; pharmacist follow-up versus usual discharge; unplanned urgent visits within 90 days.” You can then check whether each paper studied the population, comparison and outcome you need.

Separate study design, risk of bias and certainty

These judgments answer different questions:

1. Design: how was the evidence generated?
2. Risk of bias: could the methods systematically distort this result?
3. Certainty: how confident are we in the body of evidence for a particular outcome?
4. Applicability: does that evidence address the population, care and decision at hand?

For randomized trials, Cochrane’s RoB 2 guidance examines randomization, deviations from intended interventions, missing outcomes, outcome measurement and selection of the reported result. The assessment concerns a particular result. An unblinded satisfaction score and a blinded adjudication of hospitalization can have different vulnerabilities within the same trial.

GRADE uses high, moderate, low and very low certainty for outcome-specific bodies of evidence. Its five reasons for rating certainty down are risk of bias, inconsistency, indirectness, imprecision and publication bias. Study design informs the assessment, but does not settle it. See Cochrane’s GRADE chapter.

For instance, trials may give consistent estimates of a short-term symptom change while offering little information on serious harms after several years. Record the outcome beside the certainty judgment and retain the review authors’ reasons. If there is no formal assessment, describe the specific concerns you found.

HE-01: a review and a trial answer different questions

Fictional study comparison. A hospital team asks whether pharmacist follow-up after discharge reduces unplanned urgent visits within 90 days. The example uses these two reports:

Report A may help the team understand knowledge after teaching. It cannot supply an estimate of urgent visits. The directness problem would remain even if all eight included studies were randomized: the measured outcome would still differ from the team’s question.

Report B directly compares the requested strategies and outcome. Concealed allocation reduces the opportunity to manipulate enrollment based on the next assignment. Complete follow-up accounts for every participant; blinded adjudication reduces the risk that knowledge of assignment influences outcome classification. Next, read the full protocol: did the groups receive different care beyond pharmacist follow-up, and what schedule and staffing did follow-up require?

Work through the effect size

The fictional trial’s event risks are 12 ÷ 120 = 10% with follow-up and 24 ÷ 120 = 20% with usual care.

• Risk ratio: 10% ÷ 20% = 0.50.
• Relative risk reduction: 1 − 0.50 = 50%.
• Absolute risk reduction: 20% − 10% = 10 percentage points, or 10 fewer people with an urgent visit per 100 over 90 days.
• Number needed to treat: 1 ÷ 0.10 = 10 over 90 days, using this point estimate and comparison.

Using the usual log risk-ratio approximation with these example counts gives a 95% confidence interval of about 0.26–0.95. That interval spans a wide range of possible relative benefits. Bias and applicability to another hospital require separate appraisal.

The Cochrane interpretation chapter explains why effect size, uncertainty and baseline risk belong together. To see the effect of baseline risk, suppose the same risk ratio of 0.50 applied where usual-care risk was 4%. The corresponding risk would be 2%, an absolute reduction of 2 percentage points and a point-estimate NNT of 50. This calculation assumes that the relative effect stays constant.

Completed appraisal record

Question: structured pharmacist follow-up versus usual discharge care; unplanned urgent visits by 90 days after medication changes.

Most directly relevant evidence in this packet: Report B. Its fictional randomized comparison records 12/120 versus 24/120 events. RR 0.50; calculated approximate 95% CI 0.26–0.95. The absolute difference is 10 percentage points at the trial’s 20% comparison risk.

Concerns: one hospital, relatively few events and no measured staffing costs, acceptability or serious harms.

Role of Report A: evidence about a knowledge outcome. It provides no urgent-visit estimate and cannot independently confirm Report B’s effect.

Next step: the service lead will compare the trial’s requirements with local pharmacist capacity and the discharge population, then review the broader evidence before proposing a service change.

Practice: repair an overstated evidence summary

Read this proposed summary against HE-01 before revealing the answer:

“The highest-level review proves pharmacist follow-up cuts urgent visits by 50 percentage points for all discharged adults. The trial confirms it is safe, so the hospital should adopt it.”

Reveal the HE-01 appraisal

The review measured knowledge, not urgent visits. The trial’s 50% relative reduction corresponds to a 10-percentage-point absolute reduction, and its population came from one hospital. Serious harms were not measured in the supplied dataset. Adoption also requires a judgment about resources, patient priorities and the remaining evidence.

Revised summary: “In the fictional trial, 10% of patients assigned to pharmacist follow-up had an urgent visit within 90 days, compared with 20% assigned to usual care. The estimate is imprecise, and the trial did not measure serious harms or staffing costs. The accompanying review measured medication knowledge; none of its studies measured urgent visits.”

HE-02: a diagnostic study with the wrong patients

Fictional diagnostic study. A clinic wants to evaluate a test in adults presenting with uncertain symptoms. An early paper compares 100 people with established severe disease against 100 healthy volunteers. It reports sensitivity of 90% and specificity of 90%.

The arithmetic may be correct, but the clinic’s diagnostic problem involves patients with overlapping symptoms, competing diagnoses and earlier disease. The early sample does not represent that problem. The FDA’s diagnostic-study guidance describes how patient selection and the disease spectrum can distort accuracy estimates.

A more relevant proposed study would recruit consecutive eligible patients at the intended point in the clinic pathway, use a justified reference standard, interpret results with appropriate blinding, and account for everyone enrolled. The appraisal should ask what happened to indeterminate tests and patients who did not receive reference testing. Recruiting more healthy volunteers would not resolve the original applicability problem.

The clinic’s record should therefore say: “Accuracy reported in selected severe cases and healthy controls; performance in our symptomatic population remains uncertain.” It should not turn “90% sensitivity” into “a positive result gives a 90% chance of disease.” Those are different quantities. Specify the intended patient population, test and reference standard in your next diagnostic evidence search.

A published trial example: SPRINT

The 2015 SPRINT trial randomized 9,361 adults at increased cardiovascular risk to systolic blood-pressure targets below 120 or below 140 mm Hg. Participants were at least 50 years old; diabetes and previous stroke were exclusions. The primary cardiovascular composite occurred at 1.65% versus 2.19% per year; the hazard ratio was 0.75 (95% CI 0.64–0.89). Treatment stopped early after median follow-up of 3.26 years. Several serious adverse events, including hypotension and acute kidney injury or failure, were more frequent with intensive treatment. Read the original SPRINT paper.

The NHLBI study record also identifies the randomized design and blinded outcome adjudicators. In your summary, record the population, composite outcome, time horizon and harms. A hazard ratio of 0.75 is not a 25-percentage-point absolute reduction.

An appraisal note could read: “This trial informs a comparison in the enrolled high-risk population. Check eligibility, measurement procedures, monitoring capacity and current guidance before applying it to a different population.” Extend this historical appraisal with a search for current recommendations using Vero’s clinical guideline search.

Copyable clinical evidence appraisal record

Use one record per important outcome or clearly defined comparison. Copy the citation and where you found each fact so another reader can check your reasoning. Leave unavailable information unresolved; the abstract alone may not answer the prompts.

Use the completed HE-01 record as an example, replacing the fictional report labels with your paper’s citation and table numbers. If a systematic review has published a certainty assessment for the outcome, cite its judgment and explanation. Identify any assessment you make yourself as your own.

Turn the appraisal into a decision record

The GRADE Working Group separates certainty of evidence from the direction and strength of a recommendation. Decision frameworks also consider the balance of effects, values, resources, equity, acceptability and feasibility. A plausible benefit does not answer those questions by itself.

The discharge team needs its eligible-patient count, the pharmacist time required, patients’ priorities and evidence on harms. Include these findings beside the effect estimate in the service proposal.

Before circulating an appraisal, check that it includes:

• A question that matches the result: the correct population, comparator, outcome and time.
• Traceable numbers: denominators, effect measure, uncertainty and any calculation assumptions.
• Reasons for concern: the method or missing information behind each judgment.
• A usable conclusion: what the evidence supports, what remains unresolved and who will investigate it.

If a research assistant or AI tool supplies a summary, open the cited source and check its population and outcome. The AI for doctors guide includes an evidence-lookup exercise about a citation that supports a narrower claim than the answer makes. An evidence level attached to a citation cannot repair that mismatch.