kyegomez / BitNet

Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch
https://discord.gg/qUtxnK2NMf
MIT License
1.55k stars 143 forks source link

is this actually working? #41

Closed fblgit closed 3 months ago

fblgit commented 6 months ago

did anyone made this to work?

i tested half dozen of models.. none of them actually worked.

Upvote & Fund

Fund with Polar

kyegomez commented 6 months ago

@fblgit what does this mean? Can you elaborate?

fblgit commented 6 months ago

i couldnt get many of the models around to work, did u manage to get this to train and do inference properly? or this is some experimental non-working code/experiment/test/poc?

Ronsor commented 6 months ago

did anyone made this to work?

i tested half dozen of models.. none of them actually worked.

No, this code does not work.

dmarx commented 6 months ago

Do any of this user's repositories?

kyegomez commented 6 months ago

i couldnt get many of the models around to work, did u manage to get this to train and do inference properly? or this is some experimental non-working code/experiment/test/poc?

what code did you run, give me errors, or the output.

Also you need to finetune the model before this works so you know.

kyegomez commented 6 months ago

did anyone made this to work? i tested half dozen of models.. none of them actually worked.

No, this code does not work.

do you have an error? please post the error

kyegomez commented 6 months ago

Do any of this user's repositories?

what error do you have exactly?

suzuke commented 6 months ago

The implementation of this binear is completely wrong, not only does it not follow the process outlined in the Bitnet paper, but it also misunderstands all the computational principles. I don't understand why it still receives so many stars.

suzuke commented 6 months ago

The issues I mentioned have been addressed in the commit 6cdb2ea998e843b454f2fbaaef73bc6bf92c305f.

kyegomez commented 6 months ago

@suzuke if it's wrong please submit a pull request

github-actions[bot] commented 3 months ago

Stale issue message