search

pliang279 / MultiBench

[NeurIPS 2021] Multiscale Benchmarks for Multimodal Representation Learning
MIT License
462 stars 68 forks source link

Did you forget switch model train/eval state? #26

Closed JesseWong333 closed 1 year ago

JesseWong333 commented 1 year ago

Hi, Thank your excellent work.

Recently, I found a small performance mismatch problem. It seems that the pytorch model is not switched to eval state when you validate.

At line 169, model.eval() is called, and a few lines later, model.train() is called.

https://github.com/pliang279/MultiBench/blob/d291e36257e3f0c17b04d5c8fe3574e347e95f3c/training_structures/Supervised_Learning.py#L169

https://github.com/pliang279/MultiBench/blob/d291e36257e3f0c17b04d5c8fe3574e347e95f3c/training_structures/Supervised_Learning.py#L177

lvyiwei1 commented 1 year ago

Hi Jesse,

Thank you so much for pointing out this problem! We have fixed the code by removing the model.train() functions in the validation function. Sorry for any inconvenience this bug may have caused.