Distorting drug trials to highlight positive outcomes 
By Editorial Board, The Guardian

Disclaimer  The information presented here is for informative and educational purposes only and is not intended as curative or prescriptive advice.

In the book, Bad Science, award-winning writer, Dr. Ben Goldacre [  MD ]  goes to lengths to show how pharmaceutical companies may intentionally or unwittingly distort research into drug trials.

Drug trials are expensive. So much so, 90% of clinical drug trials, and 70% of trials reported in major medical journals, are conducted or commissioned by the pharmaceutical industry, Goldacre notes (p.204). He adds that a key feature of science is being able to replicate findings, but adds a concern that when only one organization is doing the funding, this feature is lost. Whatever the reason, he says, “the upshot is that drug companies have a huge influence over what gets researched, how it is researched, how the results are reported, how they are analyzed, and how they are interpreted.”

Although here is a brief summary, it is worth reading chapter 11 from Goldacre’s book to get a detailed idea of how all the above can happen and how distorted drug trials can become.

Goldacre describes numerous ways in which drug trials can be conducted that will distort results from the start. Examples include:

    061  Study the drug on winners. That is, choose people who are likely to show improvements. This, Goldacre says, will make the research less applicable to the people doctors would intend it for, but he adds sarcastically, “hopefully they won’t notice.” Worryingly, this is so commonplace, that it’s hardly worth giving an example, he adds. (But Goldacre also points readers to a book now available online, for free, called Testing Treatments; Better Research for Better Healthcare which details this and many of the other issues in more detail).
    061  Compare the drug against a useless control. For example, rather than comparing against the best drugs currently available, comparing a new drug to a placebo is almost always guaranteed positive results that can be talked about.
    061  If comparing against a competitor’s drug, try to use the other drug unfavorably. For example, give a smaller dose of the alternative drug (so patients don’t do very well), or a higher dose (so that there are more side effects). Or, give the drug in an inappropriate way (e.g. orally when it should be intravenous).
    061  Use a surrogate outcome rather than a real outcome to measure against. Goldacre’s example: “if your drug is supposed to reduce cholesterol and so prevent cardiac deaths … don’t measure cardiac deaths, measure reduced cholesterol instead. That’s much easier to achieve than a reduction in cardiac deaths, and the trial will be cheaper and quicker to do, so your result will be cheaper and more positive.”
    Once the trial is done, how can the results be dealt with? Goldacre continues:
      061  If generally good results, don’t draw attention to negative aspects in the graphs. Maybe mention in the text briefly.
      061  If results are negative, don’t publish them, or publish after a long delay.
      061  Even manipulate the statistics (which Goldacre details from p. 209 to 211).
      061  Some positive tests, but under conditions of unfair trials, may even be recorded with obscure journals. Hopefully only the abstract will be read.

This is of course scandalous and hard to believe. Yet, Goldacre describes at length how this can come about through various factors.

For example, “publication bias” means that mostly positive results get published, and so it is in the interest of researchers to highlight and push those. Constantly getting negative results can be quite demotivating for researchers, even though it is actually important to know when something does not work. Other times, there can be pressure to not publish negative results (as also discussed in more detail further below).

It may seem as not harmful, but as Goldacre adds, “Doctors need reliable information if they are to make helpful and safe decisions about prescribing drugs to their patients. Depriving them of this information, and deceiving them, is a major moral crime.” (pp. 215 – 216)

“Duplication bias” is another problem: the same positive result gets republished in many places, in different forms, so it looks as if there are lots of different positive trials.

Addressing many of these concerns is conceptually simple, according to Goldacre:

Almost all of these problems — the suppression of negative results, data dredging, hiding unhelpful data, and more — could largely be solved with one very simple intervention that would cost almost nothing: a clinical trials register, public, open, and properly enforced. This is how it would work. You’re a drug company. Before you even start your study, you publish the “protocol” for it, the methods section of the paper, somewhere public. This means that everyone can see what you’re going to do in your trial, what you’re going to measure, how, in how many people, and so on, before you start.

The problems of publication bias, duplicate publication and hidden data on side-effects — which all cause unnecessary death and suffering — would be eradicated overnight, in one fell swoop.… There are trial registers at present, but they are a mess.

— Ben Goldacre, Bad Science, (Harper Perennial, 2009), pp.220 – 221 (emphasis is original)

Finally, Goldacre describes how Pharmaceuticals ultimately distort things: through advertising. In some countries, such as UK, advertising directly to patients is not allowed. In others, such as the US, it is:

Patients are so much more easily led than doctors by drug company advertising that the budget for direct-to-consumer advertising in America has risen twice as fast as the budget for addressing doctors directly. These adverts have been closely studied by medical academic researchers, and have been repeatedly shown to increase patients’ requests for the advertised drugs, as well as doctors’ prescriptions for them.

… This is why drug companies are keen to sponsor patient groups, or to exploit the media for their campaigns.

— Ben Goldacre, Bad Science, (Harper Perennial, 2009), p. 222

He goes on to describe a case where Britain’s NHS was pressured to approve a drug for Alzheimer’s even though the evidence for its efficacy was weak and because drug companies had “failed to subject their medications to sufficiently rigorous testing on real-world outcomes … that would be much less guaranteed to produce a positive result.”

Media headlines then make it look like the “death panels” that have been conjured up by some in recent US health reform debates, ignoring these more, seemingly boring, details.


Goldacre Ben, "Is the conflict of interest unacceptable when drug companies conduct trials on their own drugs? Yes," BMJ, November 29, 2009. Goldacre: conflcits of interest

Goldacre Ben, "Ghostwriters in the sky," The Guardian, September 18, 2010.   Ben Goldacre: bad science