search

kyegomez / BitNet

Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch
https://discord.gg/qUtxnK2NMf
MIT License
1.55k stars 143 forks source link

forward method in Class BitLinear #6

Closed guoqixin1 closed 11 months ago

guoqixin1 commented 11 months ago

hello, thanks for your Implementation. I was a bit confused while reading the bitnet/bitlinear.py forward() as the paper shown:

企业微信截图_09dcb834-cb93-4cd5-8d02-84d33f63a955

i think the forward method should be:

image

did i misunderstand the process?

kyegomez commented 11 months ago

@guoqixin1 Hey thanks for pointing this out, we changed it yesterday so now there is a new verison let me know if you find any new issues