Frequentist visualisation - early stopping p-value imputing messes up confidence interval

Richie-Lee / Bayesian_AB_Test

MSc_Thesis code repository

0 stars 0 forks source link

Open Richie-Lee opened 6 months ago

Richie-Lee commented 6 months ago

option 1: calculate ALL p-values, even for early stopped -> seems like best option, but requires modified code logic and longer compute times for large n
option 2: only calculate confidence interval over non-early-stopped runs

Not sure whether visualisation is important anyway

Richie-Lee commented 6 months ago

Decision - leave this shit in there, if the visualisation becomes important we will take option 1