search

allenai / OLMo

Modeling, training, eval, and inference code for OLMo
https://allenai.org/olmo
Apache License 2.0
4.2k stars 392 forks source link

Fixes clipping #601

Closed ananyahjha93 closed 1 month ago

ananyahjha93 commented 1 month ago

Added tests (CPU and GPU) to compare torch clipping and olmo clipping and fixed clipping for DDP and FSDP no_shard

@epwalsh can we merge this PR so that I can push the DDP one after this?