Closed stevenhurwitt closed 4 years ago
You are in luck. The code can do it. That's why we have a list-mode primary data format.
Gallas, B. D. & Brown, D. G. (2008), 'Reader Studies for Validation of CAD Systems', Neural Networks Special Conference Issue 21(2), 387-397.
Gallas, B. D.; Pennello, G. A. & Myers, K. J. (2007), 'Multireader Multicase Variance Analysis for Binary Data', J Opt Soc Am A, Special Issue on Image Quality 24(12), B70-B80.
Thanks! I will look into it in the next few days - the naive way I tried was giving AUC's of around .5 and I assume averaging over all readers (they should have been around .9 with the sample data I was using)
Is there any way to manipulate the code to work with a non-fully crossed design? For instance if readers are divided up across the cases (and every reader does not read every case), then the AUC values are only a fraction of what they should be.
Is there any literature on how to adjust for this, or any functionality within the current code?