SSL pre-training and fine-tuning on `(64, 64, 64)` samples

[valosekj]

This issue summarizes the follow-up experiment to https://github.com/ivadomed/model-seg-dcm/issues/7.

Pre-training

In https://github.com/ivadomed/model-seg-dcm/issues/7., the entire input image of size (64, 256, 256) was used to create two augmeted views. Here, two random samples of size (64, 64, 64) are taken from the input image to create the augmented view using these transforms. Moreover, these samples are taken around a label (spinal cord mask) using RandCropByPosNegLabeld:

[valosekj]

I tried three experiments, all for 500 epochs, modifying:

• training batch_size
• number of holes during samples augmentation

batch_size=8, 5 holes

Reconstructed images across epochs: link

👉 visually looks the best, training now for more epochs

loss_plots


batch_size=8, 10 holes

Reconstructed images across epochs: link

loss_plots


batch_size=16, 10 holes

Reconstructed images across epochs: link

loss_plots


[valosekj]

Two experiments with RandCoarseDropoutd augmentation transform: fill_value=None vs fill_value=0. Discussion here. Both used batch_size=8, 5 holes, and 1500 epochs.

RandCoarseDropoutd fill_value=None

= filling the dropped regions with randomly selected value based on the min and max value of input image

Reconstructed images across epochs: link

loss_plots


RandCoarseDropoutd fill_value=0

= filling the dropped regions with value 0

Reconstructed images across epochs: link

loss_plots


[valosekj]

Fine-tuning on dcm-zurich-lesions

Since the pre-training seems to work (at least the code does not raise errors and the model is learning to reconstruct the images), I performed two fine-tuning experiments: using a pre-trained model vs training from scratch.

Pre-trained model

Fine-tuning using the SSL pre-trained model with RandCoarseDropoutd fill_value=0 (see comment above).

finetune loss plots --> crashed to zero


Although the learning crashed, the model was learning something! See the validation figure below.

validation figure


Training from scratch

Fine-tuning from the scratch (weights were initialized randomly).

loss plots --> crashed to zero