search

mozilla / DeepSpeech

DeepSpeech is an open source embedded (offline, on-device) speech-to-text engine which can run in real time on devices ranging from a Raspberry Pi 4 to high power GPU servers.
Mozilla Public License 2.0
25.29k stars 3.96k forks source link

Hyper parameter schedules #2988

Open tilmankamp opened 4 years ago

tilmankamp commented 4 years ago

Support for hyper parameter schedules during training. They could get specified as comma-separated value sequences:

python DeepSpeech.py ... --dropout_rate 0.40,10:0.30,50:0.20 ...

This example also shows how colon-separated prefixes could be used to specify the epochs at which the prefixed values should get reached.

reuben commented 4 years ago

Someone said they were interested in contributing a LR scheduler a while ago, I don't remember who it was though :/

reuben commented 4 years ago

Also, this would be a good opportunity to refactor the training code a bit and make it more reusable so that logic like this can be implemented as an orchestration of existing functions rather than hardcoded into the training loop. For example if individual training steps and epochs are simple enough to set up and call from an external, simpler training script, the script can contain all of the schedule parsing logic, instead of it being spread throughout the code.

nilabja-bhattacharya commented 4 years ago

Hi, I would like to work on this. I am a new contributor, please let me know the files to get started.