search

Liuhong99 / Sophia

The official implementation of “Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training”
MIT License
938 stars 52 forks source link

why did you delete SophiaH? #53

Closed Andron00e closed 1 month ago

Andron00e commented 1 month ago

Hello, dear authors! Why did you delete SophiaH? Sincerely, yours

Liuhong99 commented 1 month ago

Could you please refer to https://github.com/stanford-crfm/levanter/blob/331c0aa02eec635fa220fc44267cede455b1bca2/src/levanter/optim/sophia.py for SophiaH?