search

mlfoundations / open_lm

A repository for research on medium sized language models.
MIT License
320 stars 41 forks source link

Fix order of loss resets. #259

Closed GeorgiosSmyrnis closed 3 weeks ago

GeorgiosSmyrnis commented 1 month ago

Due to the ordering of the loss resets, training would not actually exit if a NaN value was encountered. This PR fixes this.