search

kyegomez / BitNet

Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch
https://discord.gg/qUtxnK2NMf
MIT License
1.55k stars 143 forks source link

Is this forward implementation equal to torch.sign(input)? #9

Closed talenz closed 11 months ago

talenz commented 11 months ago

https://github.com/kyegomez/BitNet/blob/979be3cd3218ebbd65c568c85e7a593d0f887a34/bitnet/straight_through_estimator.py#L10

kyegomez commented 11 months ago

Basically yes @talenz