kyegomez / BitNet

Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch
https://discord.gg/qUtxnK2NMf
MIT License
1.69k stars 155 forks source link

need a distributed training example #17

Closed sosofun closed 8 months ago

sosofun commented 10 months ago

Thank you for your innovative work, can you provide a distributed training example?
then can quickly reproduct and verify thesis work。

Upvote & Fund

Fund with Polar

kyegomez commented 10 months ago

@sosofun yes I can

github-actions[bot] commented 8 months ago

Stale issue message

kyegomez commented 8 months ago

@sosofun try it out! it's been created