The Lancet paper by Shang et al.
In August 2005, the Lancet published a study1 which was reported as having compared 110 similar trials on homeopathy and conventional medicine, and reached the conclusion that homeopathy is no better than placebo.
An accompanying editorial entitled ‘The end of homoeopathy’2 lead to widespread media attention.
However, the conclusions of this study were based on only 8 of the 110 trials, none of which tested usual homeopathic care. Furthermore, if just one of the 8 trials they picked from the 110 is changed, the results are reversed, showing that homeopathy works beyond placebo.3
This demonstrates that the findings of this paper are completely unreliable.
The sequence of events surrounding this issue of The Lancet journal, the political controversy which followed and key flaws of the Shang study itself, are summarised in ‘Homeopathy and the Lancet‘.3
A paper with continued global impact
The Shang et al. paper has become notorious because it continues to be misquoted to this day as definitive ‘proof’ that homeopathy doesn’t work, despite its fundamental flaws. For example, Ben Goldacre, British physician, academic and science writer stated (Pharma Chameleon’, New Europe, Issue 93, 17 April 2011):
“We’ve had over 100 trials of homeopathy now. For any medical treatment, after 100 trials have failed to give good overall evidence of benefit, this is the point at which a sensible person, with no vested interests in that one particular treatment, would loudly and clearly state: “No more money should be spent researching this blind alley”.
Shang et al. key facts
- Although 110 homoeopathy trials and 110 matched conventional medicine trials were included in the study, the conclusions were based on only 8 homeopathy trials and 6 conventional medicine trials described as ‘larger higher quality trials’.
- If all 21 homeopathy trials they identified as higher quality had been analysed, they would have got the opposite result – that homeopathy has an effect beyond placebo.10 No justification has been given for discounting the other 13 high quality trials just because they were smaller.
- The study fails sensitivity analysis i.e. if just one of the 8 homeopathy trials they chose from the 110 is changed, you get the opposite result, that homeopathy works better than placebo.4
- All 8 trials tested non-individualised homeopathy i.e. the same remedy given to all patients. The conclusions of this study therefore have no relevance to usual homeopathic clinical practice involving individually prescribed medicines, given after an in-depth consultation.
- The study does not state which 8 homeopathy trials and which 6 conventional trials they used. Such lack of reporting transparency is typically not tolerated in studies of conventional medicine.6 Following an outcry, the authors eventually provided this information.
- The Shang paper, containing studies published up to 2003, is now out of date and superseded by more recent and robust systematic reviews.13
Results of the Shang paper
Shang et al.’s findings as reported in the abstract of the paper are:
110 homoeopathy trials and 110 matched conventional-medicine trials were analysed. The median study size was 65 participants (range ten to 1573). 21 homoeopathy trials (19%) and nine (8%) conventional-medicine trials were of higher quality. In both groups, smaller trials and those of lower quality showed more beneficial treatment effects than larger and higher-quality trials. When the analysis was restricted to large trials of higher quality, the odds ratio was 0·88 (95% CI 0·65–1·19) for homoeopathy (eight trials) and 0·58 (0·39–0·85) for conventional medicine (six trials).
How reliable are these results?
The role of the Shang et al. study in the homeopathy debate cannot be overstated, so its quality and reliability are paramount.
Unfortunately, the poor scientific quality of the paper is obvious, and has been described by experts in the field of homeopathy research3,4,5,6,7 as well as independent researcher Prof Hahn, who highlights the main problem:
To conclude that homeopathy lacks clinical effect, more than 90% of the available clinical trials had to be disregarded.8
However, Prof Hahn from Sweden has ‘never practiced, received, or studied homeopathy, but has worked in clinical medicine and performed traditional medical research in anesthesiology and surgery for the past 30 years’. His paper details why the results of the Shang paper, which includes trials on many different diseases, would not usually be accepted in scientific circles:
The ultimate argument against homeopathy is the ‘funnel plot’ published by Aijing Shang’s research group in 2005. However, the funnel plot is flawed when applied to a mixture of diseases.8
Shang et al. 2005 paper is now woefully out of date
Reliability of the analysis is not the only problem with the Shang paper. As this meta-analysis only includes studies published up to 2003, it can no longer be seen as relevant literarure, having been superseded by more recent papers, particularly the systematic review programme led by Dr Mathie.
For example, the Mathie et al. 2014 study found that homeopathic medicines, when prescribed during individualised treatment, are 1.5 to 2.0 times more likely to have a beneficial effect than placebo.13
