Faster Training Process for MONAI Label

[finalelement]

This is a list of techniques for faster convergence to complete the training process faster for Spleen/Heart.

Experimental Settings observed for trial are:

• Novograd Optimizer instead of Adam
• Higher learning rate of 0.0002 vs 0.0001
• Dice CE Loss with squared predictions and batch set to true

@SachidanandAlle If you have another issue in parallel for the thread data loader, please direct me to the correct milestone and Ill tag this one there as well.

I will report here with experimental findings.

[SachidanandAlle]

can you prepare a PR for these changes?

[finalelement]

PR raised: https://github.com/Project-MONAI/MONAILabel/pull/485

Identified room for improvement by usage of DiceCE Loss function, Novograd optimizer. I propose we add these with an option for fast training flag.

Two other things up for consideration are ThreadDataLoader's and the usage of transform ToDeviced (allows for moving data to the GPU for faster training).

[SachidanandAlle]

I am working on this.. will add options to select ThreadDataLoader and provision user to run some transforms on GPU

[SachidanandAlle]

https://github.com/Project-MONAI/MONAILabel/pull/485 solves this problem

For example the training per epoch is reduced to 4 sec from 16 sec (keeping the hardware constant)