From early RCT: effect of Remdesivir on mortality could go either way

Consider the recent randomized double-blind, placebo-controlled trial of Remdesivir for patients with severe COVID-19. The study found no discernable difference in mortality: Twenty-two of \(158\) (\(14\%\)) Remdesivir patients died within \(28\) days while \(10\) of \(78\) (\(13\%\)) in the placebo group died (Table 1). How different would the results have to be in the current study to change statistical inference about Remdesivir?

Table 1. Comparison of Remdesivir vs Placebo Control on Mortality. Data from Table 1, Wang et al (2020). Cells represent number of cases by day 28 of study. RIS=Robustness of Inference to Switches.

Table 1. Comparison of Remdesivir vs Placebo Control on Mortality. Data from Table 1, Wang et al (2020). Cells represent number of cases by day 28 of study. RIS=Robustness of Inference to Switches.

To answer this question, we identify the number of Remdesivir treatment patients that would need to be switched between “died” and “survived” to exceed an effect size of a given magnitude – a quantity we refer to as the Robustness of the Inference to Switches (RIS). We calculate that the effect would be statistically significant and positive if \(14\) or more of the \(22\) treated patients who died instead had survived (\(RIS=-14\)). Figure 1 places these results in a larger context by plotting the change in effect size as a function of switching outcomes for treatment cases. The estimated effect from the observed data is \(14\) switches below the top dashed line representing statistical significance at the \(.05\) level for a positive finding, and \(16\) switches above statistical significance at the \(.05\) level for a negative finding (bottom dashed line). The RIS shows that even though the estimated effect is near zero, a change in outcomes for a small absolute number of patients (\(14\) or \(16\)) could generate a different inference in either direction.

Figure 1. Decrease in probability of mortality for treatment group after hypothetically switching treatment survival cases to treatment death.Black square, study estimate; top dashed line, positive estimate significant at 5\% level; bottom dashed line, negative estimate significant at 5\% level.

Figure 1. Decrease in probability of mortality for treatment group after hypothetically switching treatment survival cases to treatment death.Black square, study estimate; top dashed line, positive estimate significant at \(5\%\) level; bottom dashed line, negative estimate significant at \(5\%\) level.

Using KonFound-it! to reproduce the results

Now let’s see how to use R package konfound to reproduce the results.

Similar to the HCQ example, we have a \(2 \times2\) table in this Remdesivir example. Accordingly, we use tkonfound function to calculates the number of cases (RIS) that would have to be switched from one cell to another of a \(2 \times2\) table to invalidate an inference made about the association between the rows and columns.

konfound::tkonfound(a = 10, b = 68, c = 22, d= 136)
## [[1]]
## [1] "The tkonfound function calculates the number of cases that would have to be switched from one cell to another of a 2x2 table to invalidate an inference made about the association between the rows and columns. This can be applied to treatment vs control with successful vs unsuccessful outcomes."
## 
## [[2]]
## [1] "See konfound_fig for full and accessible details in graphic form!"
## 
## [[3]]
## [1] "To sustain an inference, 16 cases need to be transferred from treatment success to treatment failure, as shown, from the User-entered Table to the Transfer Table."
## 
## $User_enter_value
##           Fail Success
## Control     10      68
## Treatment   22     136
## 
## $Transfer_Table
##           Fail Success
## Control     10      68
## Treatment   38     120
## 
## [[6]]
## [1] "For the User-entered Table, we have a Pearson's chi square of 0.054, with p value of 0.816."
## 
## [[7]]
## [1] "For the Transfer Table, we have a Pearson's chi square of 4.065, with p value of 0.044."
library(konfound)
## Sensitivity analysis as described in Frank, Maroulis, Duong, and Kelcey (2013) and in Frank (2000).
## For more information visit http://konfound-it.com.
konfound::tkonfound_fig(a = 10, b = 68, c = 22, d= 136)
## [[1]]

## 
## [[2]]
## [1] "A bend in line indicates switches from the control row because the treatment row was exhausted."
## 
## [[3]]

 
comments powered by Disqus