Aiming to Fix Conformer OOM

mlcommons / algorithmic-efficiency

MLCommons Algorithmic Efficiency is a benchmark and competition measuring neural network training speedups due to algorithmic improvements in both training algorithms and models.

https://mlcommons.org/en/groups/research-algorithms/

Apache License 2.0

321 stars 62 forks source link

Aiming to Fix Conformer OOM #710

Closed pomonam closed 5 months ago

pomonam commented 5 months ago

Tries to address #663.

Resets the memory at the beginning of training.
Change dropout to have inplace=True

github-actions[bot] commented 5 months ago

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

priyakasimbeg commented 5 months ago

Hey Juhan, can you address the lint errors so we can merge this in?