As one of the forefathers of evidence-based medicine, how and why did you get stated in the field?
I was a resident in Internal Medicine at McMaster University and I was very fortunate because [Dr.] David Sackett and other senior people in the department, people like [Dr.] Peter Tugwell and [Dr.] Brian Haynes, were already here and had started a new department—the first in the world—of clinical epidemiology. They were starting to teach people, clinicians, to use the medical literature to optimize their patient care. They called it “critical appraisal” at first, and it started as kind of a classroom activity. You went to a classroom, away from your clinical care, and learned about the principles of understanding and using medical literature. Then, David Sackett came up with the concept of bringing “critical appraisal” to the bedside, which was to take it right where you are delivering care and using the literature in the very process of it. In 1990, I took over the Internal Medicine residency program [as its] director, and my notion was to create the world’s first program that would implement this idea of bringing understanding of the evidence to the direct care of the patients. I needed a name for what I was going to call this, because I wanted to say, “Okay, if you are interested in doing Internal Medicine residency and you want to come to McMaster, come and you will do ‘something,’ and I needed a name for it. My first name [for it] was ‘scientific medicine’ which was rejected with tremendous hostility by my colleagues in the Department of Internal Medicine. And so, “evidence-based medicine” was my second idea for what to call this, and it turned out to be a very catchy name.
There has been criticism about evidence-based medicine regarding information overload and vested interests of pharmaceutical companies. What are your thoughts on this and how do you think these problems can be best combated?
With respect to the pharmaceutical industry, what we need to do is to train clinicians and the various sorts of experts who [make] guidelines for clinicians to be aware of the spin that is put on pharmaceutical industry studies. The pharmaceutical industry does its studies very well in terms of concealing randomization, blinding everybody possible, managing complete follow-up, and ensuring accuracy of the data. So the methods and the data that come out of [these] studies [are] generally very high-quality. The problem is how they interpret and use the data and that problem is solved by having [a third-party interpret it] for clinicians to be able to understand the appropriate analysis of that data.
In terms of information overload, few clinicians are going to have both the skills and the time to actually critically read the original literature and use it in their patient care. So what they are going to have to understand [is how to use available results to determine] the benefits and harms of [the] interventions they are thinking of using. But, they need a summary of the information from somebody else and so fortunately, now we have many institutions providing clinical practice guidelines [and] we have many systematic reviews —these are two ways that clinicians can get good evidence summaries. There is another way, and I need to state my conflict of interest here because, for a number of years, I have done a lot of consulting/education work with a world-leading electronic medical textbook, Up -To -Date. I’m biased because my job is to get them to be as evidence-based as possible, so after working with them for a decade, I’d better believe we’ve made some progress. [This] is another source of good pre-processed information that clinicians can use that ideally will direct them to the best evidence and then evidence summaries that allow them to understand the benefits and harms that the studies portray.
How can you encourage people to adopt the GRADE system and would that help combat the aforementioned problems with evidence-based medicine?
I think GRADE has been a big step forward for [both] clinical epidemiology and evidence-based medicine. It is crucial to be able to distinguish evidence you can trust from evidence that is untrustworthy, though that’s probably too strict of a dichotomy. There’s evidence that you might call “high-quality,” “moderate-quality,” “low-[quality],” and “very low-[quality],” the “very low” being very untrustworthy. And if we are going to get it right, we need to know whether our evidence is “high,” “moderate,” “low,” or “very low” quality and we need a system that is well-developed, carefully thought out, transparent, and has rules that allow people to apply the system effectively, and GRADE has done all of that.
The GRADE working group started to meet in 2000, [and] put out its first guidance in 2004. It has put out many papers since then, clarifying and deepening the guidance, and it is now a system that is used by over 110 organizations worldwide, including “Up-To-Date.” It provides a uniform, transparent, and standard way of deciding what is more trustworthy and what is less trustworthy evidence, which is a big step forward.
You recently published a study investigating the consumption of red meat that has been somewhat controversial. Do you mind elaborating a bit on the study’s findings and why you think it has received the criticisms that it has?
What we did was to look at all the evidence on the health effects of red meat. There were four systematic reviews that we published. One focused on observational, non-randomized studies of people who ate more or less red meat and its effects on [developing] cancer and cardiovascular disease, and looked at dietary patterns that included more and less red meat. As well, another review [of ours] looked at all the trials where people were randomized to diets that had more red meat or less red meat. The results of these were very clear, and were quite consistent with prior reviews, although prior reviews did not present the data optimally, which I will explain in a second. So what we found was that the studies do show an association between red meat and cancer, and red meat and cardiovascular disease, but the association is… very weak. To give you an example, we’re convinced appropriately that smoking causes cancer, because in heavy smokers, it increases your risk of cancer ten-fold. Here, the magnitude of association was increases of less than 20%, so very small associations in relative terms. [There are] biases that are possible in observational studies, because people who eat red meat are different than people who don’t eat red meat in ways other than [their red meat consumption]. So the diets that contain red meat are different from the diets that don’t in other ways. People might smoke more, or [have] different socioeconomic statuses, or be exposed to toxins —[there are] lots of differences. These observational studies generally lead to low-quality evidence, unless there are very strong effects, such as in smoking, and that was not there at all [in our study]. Moreover, the most convincing, randomized, large-scale trial available that looked at people who ate more or less red meat did not show any differences in cancer or cardiovascular disease. Now this study has limitations too, probably [involving] low-quality evidence. Bottom line, we only have low-quality evidence [to work with]. Even if we accept that there is a causal relation —which is not at all certain —but even if we accept that the effect is very small for individuals, you would have to cut your meat consumption by three servings a day for the rest of your life to cut your cancer mortality risk, best estimate, by 7 in 1000. Most people would say that is a very small effect. So the bottom lines are, [there is] only low-quality evidence and even if there is an effect —which there may not be —the effect is very small, and the magnitude of gain you would have in cutting your red meat is [also] very small.
We then did another systematic review, which looked at people’s feelings about their meat [consumption], what we call their values and preferences. Perhaps not surprising, we found that people who eat meat have lots of reasons for doing it that they think are good, and that they might not be at all keen to reduce their red meat consumption substantially for very uncertain and small benefits. So, our guideline panel said, “Okay, so we now want to make a recommendation,” and the recommendation is based on this: what do we think that most people [would do], if they understood the evidence and they were red meat eaters? Would they choose to reduce, or would they not? And what our panel believed is that the majority of people who were informed of the evidence would say, “…very small effects and uncertain? Thanks for the information, [but] I’ll keep eating my meat.” And so, we made what is a weak recommendation that people continue their current red meat consumption. Why a weak recommendation? A weak recommendation in the GRADE system that we used means that the majority of fully informed people would choose the panel’s recommended action if the panel had gotten it right but a minority would not. We thought that although the majority of fully informed people would choose to continue their red meat consumption, a minority would not. They would say, “Okay, it may be a very small effect, and uncertain, but my priority is my health and I’m going to cut down on my red meat.”
Why did people get upset at [our recommendation]? People got upset at it because previous recommendations had all been “cut down on your red meat!” Frankly, if somebody says “Hey, wait a minute, let’s look at the evidence more carefully to see [if] that’s really legitimate,” it’s threatening to people who have come out and said that everybody should cut down on their red meat.
How has evidence-based medicine changed over the years? Where do you see evidence-based medicine heading in the next few decades?
When we started, we were thinking in a way that turned out to be very unrealistic —that every physician would have the time and skills to read original papers, and the methods and results of those papers. That turns out to be a completely unrealistic idea. We still think there are certain things worth educating physicians about in reading papers and particularly understanding the results. They need to understand what the benefits and downsides of interventions are, for instance. They need to understand the uncertainty of the evidence, because [it could be] low-quality evidence where they really don’t know, or high-quality evidence that you can be confident of. So clinicians have to be taught [how to assess evidence]. In my educational practice, I’ve shifted from assessing risk of bias in studies, which is [still] worth knowing about, [towards what you can understand from] the results: what are the magnitudes of the effects, what are the certainties, and on what basis do people make decisions on the certainty of evidence. So that’s been one big change that’s happened.
The other thing is, at the beginning, if you read [the guidelines] we wrote, values and preferences barely appeared at all. You know, “Oh what’s the evidence,” as if, naively, the evidence told you what to do. But, as has happened in other parts of this conversation, most of the time, evidence doesn’t tell you what to do. The right thing to do is different for different people depending on their values and preferences. So there has been a major shift to acknowledge and build the issue of people’s values and preferences into everything we do. So that is the second major shift since the beginning.
As to the future, what we need to do is to build on our understanding that many decisions are value- and preference-dependent. In other words, in the same circumstances, the values and preferences of different people will [cause them to] make different choices. And the challenge now is to make sure the choice is correct for every individual. How do we do that? We do that in what has been called “shared decision-making” which is working with the patient, ensuring they understand what the options are, the evidence behind it, whether it is low-or high-quality, and the best guess as to what the benefits and downsides are. That turns out to be very challenging. To do it well, I think we will need what’s called “point-of-aid decision aid,” electronic decision aids where the physician and the patient are looking at a computer or iPad or some sort of screen that tells them a summary of the benefits and downsides. To do shared decision-making most efficiently and effectively, we’re going to need a lot of such decision aids [that are] well-developed so that they are accurate, give the right information, do it simply [and] clearly, and are actually fun for both the patient and clinician. And that’s the challenge for us in the future.
Written by Daniel Rayner1 and Hargun Kaur2