search

asappresearch / sru

Training RNNs as Fast as CNNs (https://arxiv.org/abs/1709.02755)
MIT License
2.1k stars 306 forks source link

Mixed Precision Training #203

Open Zadagu opened 2 years ago

Zadagu commented 2 years ago

Hi,

first of all I want to thank you for your great work. I'm using SRUs for speech enhancement, they do very well on a reasonable computational cost.

I would like to know if there is a possibility to train SRUs in mixed precision mode? I tried to enable it, by setting precision=16 in the pytorch lightning trainer, but that didn't do the trick.

Kind of regards, Zadagu

taolei87 commented 2 years ago

@Zadagu SRU / SRU++ can work with pytorch native mixed precision training.

See this for example: https://github.com/asappresearch/sru/blob/3.0.0-dev/experiments/srupp_experiments/train_enwik8.py#L250