search

IBM / molformer

Repository for MolFormer
Apache License 2.0
244 stars 42 forks source link

Why are you using dropout for validation steps? #8

Closed kosonocky closed 1 year ago

kosonocky commented 1 year ago

https://github.com/IBM/molformer/blob/1ae8d842695d9b412fb937385ef93f05a359162e/finetune/finetune_pubchem_light.py#L251C1-L252C1

AFAIK, dropout shouldn't be use to get validation stats on your model, and you're using 0.1 dropout for this. Could you elaborate? I may be missing something.

subhalingamd commented 1 year ago

Hi. I thought I could write a brief answer in case you are still looking for it.

Dropouts are not used in .eval() mode and pytorch-lightning runs validation steps in eval() mode under the hood. so its equivalent to not having them.

Hope it helps! Cheers.

kosonocky commented 1 year ago

@subhalingamd Wow I had no idea, thanks for clarifying. Makes total sense.

Cheers