Linear warmup learning rate is 0 when step = 0

google-research / big_transfer

Official repository for the "Big Transfer (BiT): General Visual Representation Learning" paper.

https://arxiv.org/abs/1912.11370

Apache License 2.0

1.5k stars 175 forks source link

Open danielamassiceti opened 3 years ago

danielamassiceti commented 3 years ago

Hi there

In the function get_lr() in bit_hyperrule.py, when step == 0, the returned learning rate is 0.0.

I wanted to check if this is a bug or intentional?