The problem with randomised controlled trials

  Published to Latest News on Oct 01, 2021

The problem with Randomized Controlled Trials (RCTs)

(Reproduced by Cora Weekes from “Functional Medicine – a systems approach” e-course by the Institute of Functional Medicine – July 2014)

There are several drawbacks to randomized controlled trials. To begin, the trials are tailored for drug testing, NOT clinical care. The main problem that RCTs introduce is that in order to ensure no systematic differences exist between the treatment group and the placebo group, the patients used are chosen from such a select population that the external validity is often profoundly limited. This means that while it may be valid to conclude that an RCT demonstrates efficacy of a drug in the group under study, under those controlled circumstances, such conclusions cannot be extrapolated to other populations; in essence, we have no idea how the drug will work in any other group or situation. Some factors that frequently limit the external validity of RCTs include:

Where the trial was performed. For example, what works in one country may not work in another.

• Characteristics of the patients under study. A trial often includes only the least sick patients whose prognosis is better than average and excludes those who have any co-existing medical conditions. These trials also often exclude key groups including women, the elderly, and non-Caucasians. Children are almost never studied in RCTs, but drugs approved after testing on adults are routinely prescribed for paediatric patients.
• Study procedures often don’t approximate the real world. In a RCT, patients may receive intensive diagnostic procedures and follow-up care that is very unlikely to occur in the real world.
• Use of shifting or clinically insignificant outcome measures. Such trials may change standards for success or report composite measures that lack clinical utility and are not used in clinical practice.
• Statistical manipulation. If a study shows that a drug, on average, increases pain tolerability by 1 point from 1 to 2 on a 10 point scale, this can be reported either as a 10% absolute improvement, or a 100% relative improvement. Or what if three- quarters of the patients showed no improvement, but one-quarter improved by an average of four points each? This drug that had no effect for 75% of patients can be said to show a 400% change in pain tolerance (for that 25% that responded).
• Conflicts of interest in funding. Trials are largely funded by industry or corporate interests, which introduces bias into which interventions are studied and how results are reported, with researchers often overlooking outcomes that are significant but don't reflect the 'focus' of the study.

Then there are the large post hoc analyses of the real world efficacy of drugs that had been shown to be effective by RCTs such as the STAR*D study of antidepressant drugs. This government-funded analysis studied a broad range of patients with depression and found that under “real world” conditions, antidepressants that had been remarkably effective in RCTs failed to result in sustained positive effects for the majority of people who receive them.

Most importantly, RCTs completely neglect the individualized nature of heath. They tell us nothing about how to create an individualized, patient centred therapeutic plan that will work for a patient with their unique combination of existing conditions, genetic influences, environmental factors, and lifestyle choices. 

Tags: health