Results of the MRMC analysis depends on the order of "ModalityID" in the FROC dataset.

[technOslerphile]

frocCr.xlsx It looks like the results of StSignificanceTesting depends on the order of ModalityID in TP and FP sheets.

library("RJafroc")
library("readxl")
library("xlsx")
x <- DfReadDataFile(fileName = "frocCr.xlsx", newExcelFileFormat = TRUE)
ret <- StSignificanceTesting(x, FOM = "AFROC", method = "OR", analysisOption = "RRRC")
ret$RRRC$ciDiffTrt

truth <- read_xlsx("frocCr.xlsx", sheet = "TRUTH")
tp <- read_xlsx("frocCr.xlsx", sheet = "TP")
fp <- read_xlsx("frocCr.xlsx", sheet = "FP")

write.xlsx(as.data.frame(tp), file = "froc1.xlsx",
                 sheetName = "TP", append = FALSE, row.names = F, showNA = F)
write.xlsx(as.data.frame(fp), file = "froc1.xlsx", 
           sheetName="FP", append=TRUE, row.names = F)
write.xlsx(as.data.frame(truth), file = "froc1.xlsx",
           sheetName="TRUTH", append=TRUE, row.names = F)
x <- DfReadDataFile(fileName = "froc1.xlsx", newExcelFileFormat = TRUE)
ret1 <- StSignificanceTesting(x, FOM = "AFROC", method = "OR", analysisOption = "RRRC")

#reorder tp and fp

tp <- tp %>% arrange(desc(tp))
fp <- fp %>% arrange(desc(fp))
head(tp)
head(fp)

write.xlsx(as.data.frame(tp), file = "froc1.xlsx",
           sheetName = "TP", append = FALSE, row.names = F, showNA = F)
write.xlsx(as.data.frame(fp), file = "froc1.xlsx", 
           sheetName="FP", append=TRUE, row.names = F)
write.xlsx(as.data.frame(truth), file = "froc1.xlsx",
           sheetName="TRUTH", append=TRUE, row.names = F)
x <- DfReadDataFile(fileName = "froc1.xlsx", newExcelFileFormat = TRUE)
ret2 <- StSignificanceTesting(x, FOM = "AFROC", method = "OR", analysisOption = "RRRC")

ret1$RRRC
ret2$RRRC

As you can see from the above screenshot, the trt1-trt0 and trt0-trt1 shows the same result. This is not intuitive and can be misleading with opposite inferences.

[dpc10ster]

Thanks for posting this; I am looking into a fix; Dev

[dpc10ster]

Fixed in da5b408633b4dde73fc58161db995a9dde8facff

• Program expected TP and FP worksheets to obey "good" ordering.
• If shuffled then unexpected results can occur.
• Added a sorting step to both worksheets.

[technOslerphile]

Thanks @dpc10ster . How will this new fix be made available in the R package? Will installing RJafroc using devtools::install_github be sufficient? I suppose the CRAN version is still the old version.