Lars-Kraemer
commented
2 months ago Hey @Malitha123,
- SGD vs. Adam: Adam can offer faster convergence but may lead to overfitting. SGD is slower but often results in better generalization.
- BatchNorm vs. InstanceNorm: BatchNorm is generally effective but can be less so with small batch sizes. InstanceNorm may be better suited for tasks with high variability or small batch sizes.
As you said, both are heavily dependent on the specific dataset and what works best needs to be experimented with. If in doubt, however, I recommend going with the nnUNet defaults.
Best, Lars
Malitha123
I've been working on MRI segmentation tasks using nnU-Net, and I've noticed that the standard configurations often utilize SGD as the optimizer. While I understand that the choice of optimizer and normalization technique can depend on the specific dataset, I'm curious if there's a significant difference in performance when using the Adam optimizer instead of SGD.
Similarly, I'd like to know if there's any observed difference in segmentation performance when using BatchNorm compared to InstanceNorm within nnU-Net.
Any insights or comparative studies on these aspects would be greatly appreciated.