First example: Pneumonia finding from initial hydroxychloroquine RCT only requires 1 ‘switch’ to invalidate

The first report of a randomized trial regarding hydroxychloroquine (HCQ) came from a study conducted at the Renmin Hospital of Wuhan University. The table below shows the association between treatment (HCQ vs conventional) and condition (improved vs exacerbated/ucnhanged). For Table 1, $$χ^2= 4.7$$, $$p = 0.03$$, and the authors concluded that HCQ was efficacious.

To quantify the robustness of the inference, we calculate the number of treatment cases that would need to be switched from “improved” to “exacerbated or unchanged” to change the inference – a quantity we refer to as the Robustness of the Inference to Switches (RIS). We calculate that if one of the 25 HCQ “improved” cases were instead classified as “excacerbated or unchanged” (RIS=1), the probability difference between HCQ and the control would drop to $$23$$ percentage points ($$\frac{24}{31} - \frac{17}{31}=.23$$) and would no longer be statistically significant ($$χ^2 = 3.5$$, $$p > .06$$).

Using KonFound-it to reproduce the results.

Since we have a $$2 \times2$$ table, we use tkonfound function to calculates the number of cases (RIS) that would have to be switched from one cell to another of a $$2 \times2$$ table to invalidate an inference made about the association between the rows and columns. This can be applied to treatment vs control with successful vs unsuccessful outcomes.

konfound::tkonfound(a = 14, b = 17, c = 6, d= 25)
## [[1]]
## [1] "The tkonfound function calculates the number of cases that would have to be switched from one cell to another of a 2x2 table to invalidate an inference made about the association between the rows and columns. This can be applied to treatment vs control with successful vs unsuccessful outcomes."
##
## [[2]]
## [1] "See konfound_fig for full and accessible details in graphic form!"
##
## [[3]]
## [1] "To invalidate the inference, 1 case needs to be transferred from treatment success to treatment failure, as shown, from the User-entered Table to the Transfer Table."
##
## $User_enter_value ## Fail Success ## Control 14 17 ## Treatment 6 25 ## ##$Transfer_Table
##           Fail Success
## Control     14      17
## Treatment    7      24
##
## [[6]]
## [1] "For the User-entered Table, we have a Pearson's chi square of 4.724, with p value of 0.030."
##
## [[7]]
## [1] "For the Transfer Table, we have a Pearson's chi square of 3.528, with p value of 0.060."

Now we use tkonfound_fig to generate the figure that plots the change in effect size as a function of switching outcomes for treatment cases.

konfound::tkonfound_fig(a = 14, b = 17, c = 6, d= 25)
## [[1]]

##
## [[2]]
## [1] "A bend in line indicates switches from the control row because the treatment row was exhausted."
##
## [[3]]