False Reassurance Based on Non-Significant Results

Douglas G. Altman

doi:10.5946/ce.2017.091

Articles

Page Path: HOME > Clin Endosc > Volume 50(6); 2017 > Article

Letter to the editor False Reassurance Based on Non-Significant Results: Douglas G. Altman^,; Clinical Endoscopy 2017;50(6):617-617.
DOI: https://doi.org/10.5946/ce.2017.091
Published online: August 18, 2017

Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK

Correspondence: Douglas G. Altman Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, OX2 6UD, Oxford, UK E-mail: doug.altman@csm.ox.ac.uk

• Received: June 7, 2017 • Accepted: July 3, 2017

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

4,224 Views
133 Download

Full Article

Download PDF

NOTES
REFERENCES

A journal article with a title that begins with “There is no …” immediately suggests that there may have been a misinterpretation of a non-significant result from a small study. Non-significant results, expressed either as a p-value greater than 0.05 or a confidence interval (CI) that includes the value representing no effect, are often falsely interpreted as evidence of equivalence [1]. What matters far more than the p-value is whether the CI includes values of clinical importance.

My attention was drawn to the meta-analysis of Amin et al. [2] because of its title. Those authors conducted a systematic review and found 9 studies that examined the outcomes of patients receiving different interventions for transmural endoscopic drainage of pancreatic fluid. Three drainage methods were considered, but, for no obvious reason, only two methods were analyzed: transmural drainage (TM) alone or in combination with transpapillary drainage.

Even if the studies included in the meta-analysis were of high quality, the wide CIs indicated that the results are compatible with a large difference in outcomes. For example, the odds ratio for recurrence was 1.49 in favor of TM, but the 95% CI was 0.53 to 4.21, which is consistent with halving or quadrupling of the risk. Only 6 studies contributed to this analysis, with a total of 34 events.

Further, 6 of the 9 observational studies were found to be of poor quality regarding the comparability of the groups, one of the most fundamental aspects of study quality. Other problems were also identified. The authors used a 9-item version of the Newcastle-Ottawa scale, as against the 7-item version they cited.

I noted that these authors also conducted the largest study included in their meta-analysis [3]. Despite being based on even less data, that study too concluded that transpapillary drainage has no benefit on treatment outcomes when performed in addition to TM.

The authors sensibly call for better, larger studies. Meanwhile, these two strategies for drainage have certainly not been demonstrated to be clinically or technically equivalent, as is wrongly claimed in the titles of both articles.

Conflicts of Interest: The author has no financial conflicts of interest.

REFERENCES

1. Stang A, Poole C, Kuss O. The ongoing tyranny of statistical significance testing in biomedical research. Eur J Epidemiol 2010;25:225–230.Article PubMed
2. Amin S, Yang DJ, Lucas AL, Gonzalez S, DiMaio CJ. There is no advantage to transpapillary pancreatic duct stenting for the transmural endoscopic drainage of pancreatic fluid collections: a meta-analysis. Clin Endosc 2017 Apr 17 [Epub]. https://doi.org/10.5946/ce.2016.091Article PubMed PMC PDF
3. Yang D, Amin S, Gonzalez S, et al. Transpapillary drainage has no added benefit on treatment outcomes in patients undergoing EUS-guided transmural drainage of pancreatic pseudocysts: a large multicenter study. Gastrointest Endosc 2016;83:720–729.Article PubMed

Figure & Data

REFERENCES

Citations

Citations to this article as recorded by

PubReader
ePub Link

Cite

CITE: export Copy Download Format; Close

Download Citation

Download a citation file in RIS format that can be imported by all major citation management software, including EndNote, ProCite, RefWorks, and Reference Manager.

Format:

RIS — For EndNote, ProCite, RefWorks, and most other reference management software
BibTeX — For JabRef, BibDesk, and other BibTeX-specific software

Include:

Citation for the content below
Citation and abstract for the content below

False Reassurance Based on Non-Significant Results

Clin Endosc. 2017;50(6):617-617. Published online August 18, 2017

DOI: https://doi.org/10.5946/ce.2017.091

XML Download

False Reassurance Based on Non-Significant Results