kyegomez / BitNet

Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch
https://discord.gg/qUtxnK2NMf
MIT License
1.69k stars 155 forks source link

ImportError: cannot import name 'BitLinear15bs' from 'bitnet.bitbnet_b158'[BUG] #34

Closed Bobby-youngking closed 8 months ago

Bobby-youngking commented 8 months ago

ImportError: cannot import name 'BitLinear15bs' from 'bitnet.bitbnet_b158'

Upvote & Fund

Fund with Polar

kyegomez commented 8 months ago

@Bobby-youngking it's not in the init, still working on it!

Bobby-youngking commented 8 months ago

@Bobby-youngking it's not in the init, still working on it!

but after i use this, the result is not limited to {-1, 0, 1},

kyegomez commented 8 months ago

@Bobby-youngking were still working on it!

Bobby-youngking commented 8 months ago

@Bobby-youngking were still working on it!

does it mean that your code still need optimization?

kyegomez commented 8 months ago

@Bobby-youngking of course. all code will always need more optimization

kyegomez commented 8 months ago

@Bobby-youngking I have updatted it for BitLinear