kyegomez / BitNet

Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch
https://discord.gg/qUtxnK2NMf
MIT License
1.55k stars 143 forks source link

need a distributed training example #17

Closed sosofun closed 6 months ago

sosofun commented 9 months ago

Thank you for your innovative work, can you provide a distributed training example?
then can quickly reproduct and verify thesis work。

Upvote & Fund

Fund with Polar

kyegomez commented 9 months ago

@sosofun yes I can

github-actions[bot] commented 7 months ago

Stale issue message

kyegomez commented 6 months ago

@sosofun try it out! it's been created