search

kkoutini / PaSST

Efficient Training of Audio Transformers with Patchout
Apache License 2.0
287 stars 48 forks source link

The meaning of "swa" #8

Open xianyi11 opened 2 years ago

xianyi11 commented 2 years ago

When use your code for training model, there is "swa": true in the config file. So, what's the meaning of "swa"?

kkoutini commented 2 years ago

SWA refers to stochastic weight averaging The implementation is here slightly modified from the pytorch lightening implementation