search

lessw2020 / Ranger21

Ranger deep learning optimizer rewrite to use newest components
Apache License 2.0
321 stars 45 forks source link

Can ranger be used for NLP transformers? #40

Closed LifeIsStrange closed 2 years ago

LifeIsStrange commented 2 years ago

e.g for improving BERT-large/XLnet ? @lessw2020 friendly ping

lessw2020 commented 2 years ago

It should do well, though I have not used it expressly on NLP. I have used it extensively for vision transformers and it performed extremely well.

LifeIsStrange commented 2 years ago

Thanks