eval in training part of linear classification code

facebookresearch / moco

PyTorch implementation of MoCo: https://arxiv.org/abs/1911.05722

MIT License

4.82k stars 794 forks source link

eval in training part of linear classification code #116

Open shuuchen opened 3 years ago

shuuchen commented 3 years ago

Hi,

I noticed that eval is used in training code. I wonder why it is used here since fc is also a part of the model, which needs to be tuned.

https://github.com/facebookresearch/moco/blob/main/main_lincls.py#L326

adambielski commented 2 years ago

Only the fc layer is trained; eval doesn't freeze the layers (require_grad=False does); eval makes the layers work in a different mode depending whether they're being trained or used for inference, which is only relevant for some layers (e.g. BatchNorm, Dropout); it doesn't have an effect on a Linear layer.