Eval mode=True during training

[Shubhendu-Jena]

Hi, thank you for the great work! I have a doubt about the code. During training, you have set eval mode to true. I'd be grateful if you could help me understand why that's so and whether there is a difference in inference results if we set the model to train mode instead.

Thanks in advance

[seva100]

Hi @Shubhendu-Jena, thank you for appreciating our work! Are you sure that we actually set eval mode to True? As far as I can see, we set the train mode to True before the epoch starts: https://github.com/alievk/npbg/blob/355294b628c79b7d3b98ebe4ecd1e03c78adb8cb/train.py#L247 Or perhaps I'm missing something?

[Shubhendu-Jena]

Hi,

Thanks for the quick response. Actually, it doesn't go to that line as args.eval_in_train evaluates to True in https://github.com/alievk/npbg/blob/355294b628c79b7d3b98ebe4ecd1e03c78adb8cb/train.py#L241

This is because in train_example.yaml, we have the following: https://github.com/alievk/npbg/blob/355294b628c79b7d3b98ebe4ecd1e03c78adb8cb/configs/train_example.yaml#L6

[seva100]

Ah ok, if you use eval_in_train: True in the train config, it'll indeed use the eval mode in train. Basically, turning this option off would enable the normal train mode. Let me see if it's better to remove this line from our train config example.

[Shubhendu-Jena]

Just to clarify, the numbers reported in the paper are with train mode, right? And have you observed any difference in the evaluation results with train mode vs eval mode?

[seva100]

I think we only used train mode for the results in the paper; @alievk do you remember by any chance?