search

k2-fsa / icefall

https://k2-fsa.github.io/icefall/
Apache License 2.0
792 stars 267 forks source link

What is the purpose of --lr-hours config in LibriHeavy recipe? #1588

Closed tienanh28122000 closed 1 month ago

tienanh28122000 commented 1 month ago

Hi everyone, in the LibriHeavy recipe, there is a --lr-hours config (with the default value of 30000) instead of a --lr-epochs like other recipes (with the default value of 3.5). However, I found that both --lr-hours and --lr-epochs configs have the same purpose in the Eden scheduler. Therefore can anyone explain to me that what is the difference between the --lr-hours config and the --lr-epochs, and why the default value of --lr-hours is higher a lot when compared to the default value of the --lr-epochs? Thank you very much!

csukuangfj commented 1 month ago

@pkufool @marcoyang1998 Could you have a look?

danpovey commented 1 month ago

They are just different ways of specifying when the learning rate starts to decrease. Originally we specified it as epochs, but later switched to hours with the idea that for very large datasets, you might not do more than one or two epochs. If you initialize with lr_hours, when you do the step_epoch() call (that's the name IIRC), you have to actually supply a number of hours rather than epochs.