search

pytorch / ao

PyTorch native quantization and sparsity for training and inference
BSD 3-Clause "New" or "Revised" License
930 stars 105 forks source link

1 bit LLM implementation #67

Closed msaroufim closed 3 months ago

msaroufim commented 6 months ago

The code is out, it's quite simple and short

Opening this so I can track how to add this to ao and make sure it works well with torch.compile(). This will likely need blackwell to perform decently

https://github.com/microsoft/unilm/blob/master/bitnet/The-Era-of-1-bit-LLMs__Training_Tips_Code_FAQ.pdf

Screenshot 2024-03-20 at 2 02 18 PM
supriyar commented 3 months ago

@msaroufim can we close this now given that we have the bitnet work being tracked elsewhere?