Some questions about infobatch at epoch0

[CKK-coder]

Hi, thank you for your great work! I applied your infobatch to my own model in the way you provided. There are some uncertain phenomena to confirm with you.

1. At the 0th epoch, the loss of the model became twice that of not using Infobatch, which I believe is abnormal. Therefore, I conducted an investigation and found that this line of code was judged as true. However, All scores initialized to 3.0 should be equal to their mean, and well_learned_mask shoule be False at 0th epoch. I want to know if my understanding is correct, or if you intentionally designed it this way (double loss at the 0th epoch).
2. What should I do with "if the learning rate scheduler is epoch-based, adjust its steps accordingly at beginning of each epoch.". For example, my model's learning rate scheduler is reduce by 0.1 times at 10, 20, and 25 epochs. What should I do if I apply infobatch to my model.

[henryqin1997]

Thank you for the question.

1. At epoch 0, the judgment should be false, so that all samples are seen in the first epoch without a drop and rescale. Initializing all scores to 3.0 was because once the version of my numpy judged all 0 initialization to be all true. Adapt this logic according to your code (to make sure samples are all seen without being dropped nor rescaled).
2. Sorry for the confusion. This should be for a step-based scheduler, as the step-per-epoch number changes each epoch. So if you don't have a step-wise learning rate scheduler call, you don't need to change it. Will update the readme.