OCEBM Levels of Evidence applied to clinical recommendations

How to assess clinical Guidelines? There are many “schools” with their methods. Today we are diving into the methods developed by Oxford Centre for Evidence-Based Medicine (OCEBM), also nicknamed the “Oxford method” for evidence appraisal.

Today we are using an example close to my heart. The first international task force in which I took part; the “EULAR recommendations for the management of psoriatic arthritis with pharmacological therapies: 2019 update”.

Psoriatic arthritis is a rheumatic inflammatory disease which affects skin, joints, fingers, nails and many other organs, requiring very specific treatment

These recommendations have both “Overarching principles” and “Recommendations”.

Overarching principles (somewhat out of the scope of this article)

Overarching principles are somewhat idiosyncratic to some guidelines (as those elaborated by the European Alliance of Associations for Rheumatology (EULAR)). These are a set of rules behind all recommendations. In other words, recommendations cannot go against the overarching principles. In this case, we only have the level of agreement, ranging from 0 to 10 (explained below). 

Since overarching principles do not have levels or grades we will jump to the list of recommendations.

Recommendations — How Guidelines are designed 

When we finally arrive at the recommendations, which try to answer specific clinical questions, we see three columns to the right, namely:

• Level of evidence
• Grade of recommendation
• Level of agreement

Before explaining each one it is important to understand the step behind each recommendation:

1. The task force chooses a set of clinically relevant questions in a PICO format;
2. A systematic literature review is performed in order to answer each individual question (following the PICO format);
3. Findings for each recommendation are assessed and weighted, turning evidence into specific recommendations
4. The final set of recommendations is presented and discussed by all task force members
5. Each element of the task force votes on the level of agreement (LOA) for each recommendation. Each participant votes from 0–10. 0= no agreement ; 10= full agreement

Recommendations — How evidence is assessed with the “Oxford method”

According to the OCEBM 2009, there are 5 levels of evidence (with further subdivisions). These levels indicate how strong is the type of study that supports a given recommendation (see table below). If for a given recommendation the task force finds studies of different levels, the final level will reflect the highest level.

For example, if there is one RCT and many individual cohort studies the recommendation would have a level 1b, because there is an individual RCT (see table below).

Oxford Centre for Evidence-Based Medicine 2009 Levels of Evidence

After assessing the level of evidence behind each recommendation it is time to “grade” the evidence. In other words, the task force would access:

• Whether the study is consistent and conclusive (inconclusive/inconsistent studies will be downgraded to Grade D);
• Whether the study provides a direct answer to the PICO question used for the recommendation or whether it is an extrapolation (the task force “exports” the rationale behind a study that answers to a somewhat similar question).

Grades of Evidence (OCEBM 2009)

Let’s go for a practical example

In this case:

• Level of evidence = 1b, meaning that this recommendation is supported by individual RTCs (we won’t get too technical here);
• Grade of recommendation = A, meaning the studies (listed in level 1) were consistent. If the studies were not consistent the grade would be downgraded to grade D; If the level 1 studies were consistent but the results were extrapolated (let’s say adapted) they would have been downgraded to grade B;
• Level of agreement (LOA) = 9,6 (0,8). Meaning the vote of all task force members regarding an agreement with this recommendation had a mean value of 9,6 and a standard deviation of (0,8) (min:0; max: 10). This shows how unanimous was the final decision. 

To sum up, using the “Oxford Method” (2009) recommendations has to be assessed regarding:

• the Level of evidence behind it — 1(a-c),2(a-c),3(a-b),4,5;
• the Grade of the recommendation — A,B,C,D;
• the Level of agreement (LOA) — ranging from 0 to 10.

BONUS topic — What changes between the 2009 and the 2011 versions of the OCEBM levels of evidence?

In 1998 the CEBM levels were created for the first time and then passed through a series of revisions (with emphasis on the 2009 and 2011 versions).

According to the CEBM website, the 2011 version brings the following updates compared to the 2009 version.

https://www.cebm.ox.ac.uk/resources/levels-of-evidence/explanation-of-the-2011-ocebm-levels-of-evidence

1. Levels are designed to be easily consulted by the busy clinician. Therefore the levels were simplified by using only numbers (from 1 to 5) without letters attached (such as “1a”, and “1c”); 
2. They were modified to represent the natural flow of a clinical encounter (diagnosis, prognosis, treatment, benefits, harms);
3.  All relevant terms are defined in an extensive glossary (accurate and understandable);
4. Screening testing merited a separate entry due to their relevance.

In the case of EULAR, their recommendations convey the “spirit” of the 2011 principles, while sticking to the 2009 classification (using letters after numbers in the level of recommendations).

Bottom line

• The “Oxford method” (OCEBM) is one of many methods used to assess the strength of clinical practice recommendations;
• Regardless of whether you agree or disagree with a given recommendation, it is essential to understand how these are built and assessed in order to contra argument in a rational and competent way;
• It is essential for all clinicians (even those who do not enjoy research) to at least understand how evidence is assessed in order to take the best clinical decision possible; 
• It has advantages and disadvantages when compared to other methods, such as the Grading quality of evidence and strength of recommendations (GRADE);
• All recommendations are prone to subjectivity (in spite of all evidence), sometimes decisions have to rely on Expert Opinion and agreement is rarely 100%.

References

Gossec L, Baraliakos X, Kerschbaumer A, de Wit M, McInnes I, Dougados M, Primdahl J, McGonagle DG, Aletaha D, Balanescu A, Balint PV, Bertheussen H, Boehncke WH, Burmester GR, Canete JD, Damjanov NS, Kragstrup TW, Kvien T, Landewé RBM, Lories RJU, Marzo-Ortega H, Poddubnyy D, Rodrigues Manica SA, Schett G, Veale DJ, Van den Bosch FE, van der Heijde D, Smolen JS. EULAR recommendations for the management of psoriatic arthritis with pharmacological therapies: 2019 update. Ann Rheum Dis. 2020 Jun;79(6):700–712. doi: 10.1136/annrheumdis-2020–217159. PMID: 32434812; PMCID: PMC7286048.

CEBM. Oxford centre for evidence-based Medicine — Levels of evidence (March 2009), 2009. Available: https://www.cebm.net/2009/06/oxford-centre-evidence-basedmedicine-levels-evidence-march-2009/

Disclosure

This article does NOT give any kind of individual medical recommendation. If you are seeking medical advice please visit a licensed physician in your country.