How to run sophia optimizer with huggingface trainer.

Liuhong99 / Sophia

The official implementation of “Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training”

MIT License

931 stars 52 forks source link

Closed Dominic789654 closed 11 months ago