Misleading outputs regarding (cumulative) number of events in survival settings and possible bug

[grlm1]

Hello, I have looked into the differences in event reporting between objects generated by getSampleSizeSurvival() and getSimulationSurvival(). My attention was specifically on determining whether the number of events is intended to be understood as cumulative or not. My findings are below, I suspect at least one bug in a summary call. Since those two functions are frequently used together, I would argue that subtle semantic differences between them can easily result in mistakes. Kind regards, Tim

library(rpact)
library(dplyr)

#define combination function
#inverse normal combination function
dIN <- getDesignInverseNormal(kMax = 2,
                              alpha = 0.025,
                              beta = 0.1,
                              sided = 1,
                              informationRates = c(0.25, 1),
                              typeOfDesign = "asOF")

#Design (theory - without SSR)
#analytical ss calculation to get an idea for required N / events
dIN_no_ssr <- getSampleSizeSurvival(design = dIN,
                                    thetaH0 = 0.9,
                                    pi2 = 0.003,
                                    hazardRatio = 0.45,
                                    eventTime = 12,
                                    accrualTime = c(0,10),
                                    followUpTime = 24)

#1. with summary
summary(dIN_no_ssr)
#From Output: Cumulative number of events 21.9   87.5
#...talks about CUMULATIVE 21.9 events at interim and 87.5 final analysis 

#2. without summary
dIN_no_ssr
#From Output:
#Number of events per stage [1]: 21.9 
#Number of events per stage [2]: 87.5 
#...does not say cumulative, instead refers to events per stage, but can be deduced that its meant as cumulative

#3. calling the eventsPerStage field directly
dIN_no_ssr$eventsPerStage
#From Output:
# [,1]
# [1,] 21.87025
# [2,] 87.48101
#...same as before, implicitly cumulative

#Now with the simulation function 
#Design (simulation - with SSR)
dIN_ssr <- getSimulationSurvival(design = dIN,
                                 thetaH0 = 0.9,
                                 directionUpper = FALSE,
                                 pi2 = 0.003,
                                 hazardRatio = 0.45,
                                 eventTime = 12,
                                 accrualTime = c(0, 10),
                                 maxNumberOfSubjects = ceiling(dIN_no_ssr$numberOfSubjects[1]),
                                 plannedEvents = c(22, 88),                   #here we enter cumulative numbers
                                 minNumberOfEventsPerStage = c(NA, 88-22),    #but not here
                                 maxNumberOfEventsPerStage = c(NA, 88-22),    #intentionally leaving min=max, so we always have the same number of events in stage 2
                                 conditionalPower = 0.9,
                                 showStatistics = T
)

#1. with summary
summary(dIN_ssr)
#From Output:
#Cumulative number of events: 22.0   66.0 
#... strongly suspect this is a bug, it should not stay cumulative in the output here or the numbers are wrong

#2. without summary
dIN_ssr
#From Output:
# Number of events per stage [1] : 22 
# Number of events per stage [2] : 66 
# Cumulative number of events [1]: 22 
# Cumulative number of events [2]: 88 
#...much better, cannot be misunderstood and also numbers are as I would expect given my input

#3. calling the eventsPerStage field directly
dIN_ssr$eventsPerStage
#From Output:
# [,1]
# [1,]   22
# [2,]   66
#... now this worries me the most. The $eventsPerStage field in the getSampleSizeSurvival object returned cumulative events,
#but the field with the very same name in the getSimulationSurvival object returns non-cumulative events

[gwassmer]

Hi Tim, Thanks a lot for your comments and pointing out the bug (#Cumulative number of events: 22.0 66.0). This will be fixed immediately. We also completely agree that the $eventsPerStage should mean the same for the getSampleSizeObject and getSimulationSurvival object. I guess the best would be to introduce a new variable $cumulativeEventsPerStage to avoid such inconsistencies. And the output should be like the one for dIN_ssr. We will do this for the next release. Again, many thanks for your findings, this is very helpful for us. Kind regards Gernot

[fpahlke]

Fixed in branch dev/3.5.1

Note: we recommend to use the helper function getParameterName() to identify the correct field name.

Example:

dIN_ssr |> 
    getParameterName("Cumulative number of events")

produces "overallEventsPerStage" (and not "eventsPerStage" - we know that this is a little bit inconsistent)

[fpahlke]

I opened the new issue #21 to follow up on the issue with variable names.