Currently, the packaged model only contains 1 fold (fold_3). I did this to reduce the filesize from 1Gb+ to ~200Mb. However, when I was comparing the single fold predictions with the 5 fold ensembled predictions, I found a pretty big difference. The input image was taken from another group (Stanford), so there is a slight domain shift even though they are TEM images with a similar resolution. I won't post the original image for privacy reasons, but here is the single fold mask (current model release):
And this is the ensemble model applied to the same image:
The input image is not shown but this second mask is much better than the first one. It still shows undersampling, but it's less severe than with the single fold model. Also, the axon shapes are much more regular. I think that for small datasets, the ensembling in nnUNet works particularly well because it contains information from all training images, and the data was very limited (5 images).
With this being said, I think it would be best to package the 5 folds, even though it gives a much bigger model.
Currently, the packaged model only contains 1 fold (
fold_3
). I did this to reduce the filesize from 1Gb+ to ~200Mb. However, when I was comparing the single fold predictions with the 5 fold ensembled predictions, I found a pretty big difference. The input image was taken from another group (Stanford), so there is a slight domain shift even though they are TEM images with a similar resolution. I won't post the original image for privacy reasons, but here is the single fold mask (current model release):And this is the ensemble model applied to the same image:
The input image is not shown but this second mask is much better than the first one. It still shows undersampling, but it's less severe than with the single fold model. Also, the axon shapes are much more regular. I think that for small datasets, the ensembling in nnUNet works particularly well because it contains information from all training images, and the data was very limited (5 images). With this being said, I think it would be best to package the 5 folds, even though it gives a much bigger model.