The fragility of statistically significant results in otolaryngology randomized trials

Publication date: Available online 22 October 2018

Source: American Journal of Otolaryngology

Author(s): Mason Skinner, Daniel Tritz, Clayton Farahani, Andrew Ross, Tom Hamilton, Matt Vassar

Abstract

Objectives

The American Academy of Otolaryngology–Head and Neck Surgery regards randomized controlled trials as class A evidence. A novel method to determine the robustness of outcomes in trials is the fragility index. This index represents the number of patients whose status would have to change from a non-event to an event to make a statistically significant result non-significant.

Methods

Investigators included otolaryngology journals listed in the top 10 of one or both of Google Scholar Metrics and Clarivate Analytics' Journal rankings. For inclusion, a randomized controlled trial needed to report a one-to-one random assignment of participants to condition, contain two parallel arms or have used a two-by-two factorial design, and report at least one statistically significant dichotomous outcome.

Results

Sixty-nine trials met inclusion criteria. The median fragility index was three events (interquartile range 1–7.5). Median sample size was 72 (interquartile range 50–102.5). Modest correlations were observed between fragility index and total sample size (r = 0.27) and fragility index and event rate (r = 0.46). Investigators found no correlation between fragility index and impact factor or Science Citation Index. In 39% (27/69) of trials, the number lost to follow-up was equal to or greater than the fragility index.

Conclusion

A median fragility index of 3 indicates that three people, on average, are needed to alter the outcomes in otolaryngology trials. This indicates that the results of two-group randomized controlled trials reporting binary endpoints published in otolaryngology journals may frequently be fragile.

https://ift.tt/2yXzkaL