Number of traning steps for each epoch

[phongnhhn92]

Hello, I am just wondering why you only use 500 steps for each epoch during training ? Is there any particular reason why you do this ? Why dont you just let the model iterate through the whole dataset like this : for iteration in range(1, len(data_loader): https://github.com/facebookresearch/synsin/blob/82ff948f91a779188c467922c8f5144018b40ac8/train.py#L91

[oawiles]

There is no particular reason. I find it's helpful when developing to have more points in the tensorboard curve. Then I can see if faster if things are training properly or not. Sorry there's not a more profound reason!

[phongnhhn92]

I am just curious because by training with just only 500 steps for each epoch then you are not covering the whole dataset. I think it is a nice trick to just iterate through a small number of steps at each epoch to see the results but when you perform the full training as reported in the paper and this repo then did you train the model with only 500 steps for each epoch or went through all of them.

This question might be not be a big issue at all but I am learning from you and I hope to get the answer ^^

[oawiles]

So it does go through the whole dataset. It's a bit funny how it works. I have a seed, which is used to choose the index whenever I get an item (https://github.com/facebookresearch/synsin/blob/master/data/realestate10k.py#L135), this changes with each epoch. Then at each training 'epoch', I reset the seed to something new (https://github.com/facebookresearch/synsin/blob/master/data/realestate10k.py#L229). Assuming the random number generator works properly (which it does), it will iterate through the dataset over time, as it will randomly select different images at each epoch.

For val, I can fix these values and so see how the model does on the same samples over time.

If you visualise the images in tensorboard you should see this.

[phongnhhn92]

Thanks ! That clarify my thoughts :D Wish you all the best @oawiles