When conducting RI for multi arm Trials, the p-values of all but the first treatment arm are false. As can be seen in the histogram of simulations, there seems to be a major issue with the way ri2 simulates statistics for the treatment arms of order 3. The simulated ATEs should be centred around 0. As they are not, the p-values retrieved are wrong.
# reprex
library(reprex)
#> Warning: package 'reprex' was built under R version 4.3.3
# three arm experiment
library(ri2)
#> Loading required package: randomizr
#> Loading required package: estimatr
library(DeclareDesign)
#> Loading required package: fabricatr
N = 100
set.seed(1)
declaration_test <- declare_ra(
N = N,
num_arms = 4)
set.seed(123)
real_T <- conduct_ra(declaration = declaration_test)
ATE_t2 = -.5
ATE_t3 = .6
ATE_t4 = 1
Y = -.2 + ATE_t2 * (real_T == "T2") + ATE_t3 * (real_T == "T3") +
ATE_t4 * (real_T == "T4") + rnorm(n = N, sd = .2)
data_test = data.frame(Y, real_T)
result = conduct_ri(formula = Y ~ real_T,
assignment = 'real_T',
declaration = declaration_test,
IPW = F,
p = 'two-tailed',
data = data_test,
sharp_hypothesis = 0)
plot(result)
When conducting RI for multi arm Trials, the p-values of all but the first treatment arm are false. As can be seen in the histogram of simulations, there seems to be a major issue with the way ri2 simulates statistics for the treatment arms of order 3. The simulated ATEs should be centred around 0. As they are not, the p-values retrieved are wrong.