Labels without mirror symmetry: mirror augmentations?

[RIMcKinley]

Hi @FabianIsensee,

I'm testing the nnUnet (3D and 2D) on brain anatomy labels. Because, for example, the left and right cerebral grey matter receive different labels, I assumed that the mirror augmentations would hurt, rather than help. However, training using nnUnet on such data seems to give decent results (very good with 3D Unet and passable with 2D Unet). I can't find any code in the repo that that detects lack of symmetry and suppresses augmentations.

Of course, the newtwork could be identifying the hemispheres and labeling them correctly, even when flipped, but I'd like to look for a more banal explanation first.

[FabianIsensee]

Hi Richard, how is it going? Have you had a good trip home? To be honest:

Of course, the newtwork could be identifying the hemispheres and labeling them correctly, even when flipped, but I'd like to look for a more banal explanation first.

This is exactly what's happening :-) But this makes it of course harder for nnU-Net to actually do the labelling, so it might make sense to disable mirroring for the left/right axis.

I don't know what axis is the left/right axis in your data, so I cannot prepare a trainer class for you that does not do mirroring in that axis. But I can provide a trainer that does not do mirroring at all. Give me 5 minutes

Best, Fabian

[FabianIsensee]

Hi, please pull again, there should be a new trainer you can use: nnUNetTrainerNoMirroring

python run/run_training.py 3d_fullres nnUNetTrainerNoMirroring Task04_Hippocampus 0 --ndet

Let me know if that improves results for you.

Best,

Fabian

[RIMcKinley]

Awesome. Running now, will give you feedback as soon as it's clear there's an improvement (I'm sure there will be)

Trip back was fine, thanks for asking :-)

Richard

[JunMa11]

@RIMcKinley Hey, are there any updates here?

I'm very interested in whether there's an improvement

[MengzhangLI]

Hi~ I have implemented 1 fold nnUNet 3D full resolution (Data Augmentation, NoDA, NoMirroring) on Task04_Hippocampus. Because I don't know where to submit labelsTs on the Internet, I will just show my results on validation set and hope it's helpful to you.

Here is the logs of the last epoch:

Normal 3D fullres (with Data Augmentation): train loss : -0.9283 val loss (train=False): -0.8392 Val glob dc per class: [0.8908553893216853, 0.8731839115577776] lr is now (scheduler) 4.800000000000001e-07 current best_val_eval_criterion_MA is 0.88460 current val_eval_criterion_MA is 0.8826 No improvement: current train MA -0.9284, best: -0.9279, eps is 0.0005

3D fullres with Data Augmentation except Mirroring: train loss : -0.9283 val loss (train=False): -0.8386 Val glob dc per class: [0.8916313600746711, 0.8715628919602462] lr is now (scheduler) 9.600000000000001e-08 current best_val_eval_criterion_MA is 0.88460 current val_eval_criterion_MA is 0.8823 No improvement: current train MA -0.9279, best: -0.9278, eps is 0.0005

3D fullres without Data Augmentation: train loss : -0.9988 val loss (train=False): -0.7600 Val glob dc per class: [0.8840835907677299, 0.8703008445624687] lr is now (scheduler) 4.800000000000001e-07 current best_val_eval_criterion_MA is 0.87690 current val_eval_criterion_MA is 0.8757 No improvement: current train MA -0.9988, best: -0.9987, eps is 0.0005

On validation set, seems like there is no difference between with or without Mirroring augmentation. Just like you said "However, training using nnUnet on such data seems to give decent results (very good with 3D Unet and passable with 2D Unet)."

I'm wondering what's going on testing set because we cannot get a conclusion before many types of experiments. ;)

Best,

[MengzhangLI]

Woops

Imma check inference results (average dice scores )on validation today, wait for a while plz...

[MengzhangLI]

Here is dice result on validation set.

Dice: label 1, label 2:

Normal 3D fullres (with Data Augmentation): Dice: 0.8967177024471666, 0.8780857787810383

3D fullres with Data Augmentation except Mirroring: Dice: 0.891672196416722, 0.8719055325780735

3D fullres without Data Augmentation: Dice: 0.8823760069053761, 0.8686237671419057

Best,

[FabianIsensee]

Thanks for sharing. Looks like mirroring is important :-)