search

OpenMOSS / CoLLiE

Collaborative Training of Large Language Models in an Efficient Way
https://openlmlab-collie.readthedocs.io
Apache License 2.0
410 stars 58 forks source link

add memory profile and fix lomo #78

Closed KaiLv69 closed 1 year ago

KaiLv69 commented 1 year ago

check dtype in the first backward for lomo