Can you please guide about no of samples in the one epoch for training a network.

[ramchandracheke]

Hi, Thanks for this wonderful github repo.

I am trying to train a model. I have total 200 cases for segmentation. I have following code.

data_augmenter = miscnn.Data_Augmentation(cycles=2, scaling=True, rotations=True, elastic_deform=True, mirror=True, brightness=False, contrast=False, gamma=False, gaussian_noise=True)

model.evaluate(training, validation, epochs=20,iterations=20)

It means in 1 epoch, 200202=8000 samples will get process. Please correct me if I am wrong.

Thanks

[muellerdo]

Hey @ramchandracheke,

thank you for your kind words and interest in using MIScnn for your research.

Advanced epoch definition

The number of iterations defines the number of batches which are processed in a single epoch. If we avoid your data augmentation object (assume we have cycle = 1), then we have to following scenario:

1 epoch = 20 iterations = 20 batches which are processed full training = 20 epochs full training = 20*20 = 400 batches

If we define that you are using a batch size of 2, then 800 images are processed.

The individual configuration of the number of iterations in an epoch allow to increase the number of batches generated in a single epoch. Normally, I would recommend to have at least 100 iterations in an epoch to give your model a little bit of "orientation time" before making the big fitting step for an epoch.

Normal epoch definition

Normally, a epoch is defined as a full run across the dataset. If you set the number of iterations to None (default), then it uses the standard epoch definition, which means all 200 samples per epoch. Then we would have with a batch size of 2:

• 200/2 = 100 batches
• 1 epoch = 100 batches
• full training = 20 epochs
• full training = 2000 batches = 4000 images

Batch queue in MIScnn

As you correctly noticed, the data augmentation object have another parameter which impacts the number of iterations. When we increase the cycle parameter to e.g. 2, we generate for each loaded sample 2 augmented images. In MIScnn, samples are loaded, then preprocessed, then augmented and then cached in a batch queue. If we fix the iteration number, then the model randomly draws from this batch queue / set. If we use the standard epoch definition as full run on all samples, then it draws all available batches from the queue until its empty.

Therefore if we define cycle to 2, iterations to None and use batch size of 2:

• We have 200 samples
• We generate 400 images with data augmentation (cycle=2)
• We pack them in 200 batches (batch size=2)
• We define 1 epoch as run over all 200 batches (iterations=None)
• We have 20 epochs for a full training

• full training = 20*200 = 4000 batches = 8000 images

Hope that I was able to give you some better insights on MIScnn batching.

Cheers, Dominik

[ramchandracheke]

Hi @muellerdo,

Thank you very much for you painstaking and lucid explanations. Now I have pristine idea about it. Therefore, if total number of samples are less than iterations then some of the samples will repeat in the training process.

Suppose we have 200 samples with 400 iterations(assuming cycle=1) in 1 epoch. This scenario is equivalent to 2 epochs in normal setting. Sorry I am asking this again because I am comparing no of epochs taken by my model to other in order to see the performance gain.

This is very helpful. Thanks a million once again.

Kind regards, Ramchandra

[muellerdo]

Hey @ramchandracheke,

I apologize for the late reply.

Thank you very much for you painstaking and lucid explanations. Now I have pristine idea about it. Therefore, if total number of samples are less than iterations then some of the samples will repeat in the training process.

Correct!

Suppose we have 200 samples with 400 iterations(assuming cycle=1) in 1 epoch. This scenario is equivalent to 2 epochs in normal setting. Sorry I am asking this again because I am comparing no of epochs taken by my model to other in order to see the performance gain.

Absolutely correct! But I wouldn't 1:1 compare these two settings due to a neural network runs various adjustments and larger weight balancing operations after each epoch. So it is important to have enough batches/iterations per epoch. But honestly, it is just the feeling from my experience. I don't have any paper in my mind which analyzed this behavior. So it would be quite interesting to compare the model performance of 100 epochs with 10 iterations vs 10 epochs with 100 iterations.

Good luck on your further research.

Cheers, Dominik

[dannyhow12]

Hi @muellerdo ,

....We generate 400 images with data augmentation (cycle=2)

With an original number of images of 200, we generate 400 images with the 8 available methods. However, if I were to only apply one of the method, example the gamma augmentation method, then how many images will be produced?

Thanks for your patience and am looking forward for your clarification

Regards, Danny