search

facebookresearch / EGG

EGG: Emergence of lanGuage in Games
MIT License
281 stars 99 forks source link

LR scheduler #201

Closed robertodessi closed 3 years ago

robertodessi commented 3 years ago

Adding support for oprimizer adaptive lr within trainer

Description

Added optimizer scheduler to trainer. Added it to the simple_autoenc game and will remove it from there if the PR is positively reviewed.

Motivation and Context

Adaptive lr can be useful in various training setups. It could be also implemented as callback and accessing the optimizer state from there (each callback gets a reference to the trainer from the parent class when calling on_train_begin.

How Has This Been Tested?

Implemented a multiplicative scheduler in simpler_autoenc and launched python -m egg.zoo.simple_autoenc.train --vocab_size=3 --n_features=6 --n_epoch=2 --max_len=3 --batch_size=32 --random_seed=21 --lr=1

robertodessi commented 3 years ago

TODO:

check model saving and loading when a scheduler is used works fine