Question

[justingm]

Hello Oliver,

Thanks for your excellent work! I tried to use the code but I have a few questions.

1. In the paper, you said that you followed a nested k-fold cross validation procedure wherein k represents the number of mice in the dataset. Does that mean that for the native CT, you did 20-fold cross validation since you have 20 mice in total? How exactly are you dividing the entire dataset into the train/validation and test sets? (Sorry if it's a stupid question.)
2. You said that you did an ensemble voting wherein you chose the median value for the final prediction. Basically, if I train it 10 times (each with different mouse as validation), I get 10 models each with its prediction and then you take the median for each pixel. Is this also included in the code? I checked it but I couldn't find it so I am assuming I have to do the post-processing myself.

Best, Justin

[OSchoppe]

Hi Justin,

Without going into the AIMOS paper in NatComms in too much detail, I am happy to try to help. Implementation details from the paper may be specific to the presented use case - and not of general interest to people using the code from GitHub in general. Thus, I try to keep discussions regarding AIMOS as a toolbox versus the science in the NatComms paper a bit separated (also see my email regarding your other questions).

1) In general, you can choose the split as you want / as useful for your application. You need to make sure though that you do not present two different scans of the same mouse during learning and during validation/testing - this would be an implicit information leak. For the results presented in the NatComms paper, we indeed used a nested k-fold cross-validation with k equal to the number of mice (not the same as the number of scans). So for k=20, the test set contains all scans of one given mouse; repeating this then k=20 times then covers the full data set

2) Again, in general you can implement this as you want / as useful for your application. Depending on your application, different implementations may be most suitable. For the ensemble voting presented in the NatComms paper, we did it exactly as you described. We opted to take the median (not the mean) as this helps to eliminate outlier predictions (say, because one model failed to converge completely; we did not really observe this but this acts mostly as a safeguard)

[justingm]

Hello Oliver,

Thanks for the clarification! I also saw your email. I'll give your suggestions a try!

Best, Justin