search

allenai / OLMo

Modeling, training, eval, and inference code for OLMo
https://allenai.org/olmo
Apache License 2.0
4.24k stars 400 forks source link

Add back original legacy unsharding implementation #562

Closed 2015aroras closed 2 months ago

2015aroras commented 2 months ago

The shared memory implementation for legacy checkpoint unsharding does not seem to be working for everyone. Adding the original implementation back in as the default, with the sharded mem implementation usable via an option to unshard.py.

2015aroras commented 2 months ago

@dwadden @OyvindTafjord FYI