Alpha-VLLM / LLaMA2-Accessory

An Open-source Toolkit for LLM Development
https://llama2-accessory.readthedocs.io/
Other
2.71k stars 176 forks source link

merge my own fork to this quantization branch #34

Closed kriskrisliu closed 1 year ago

kriskrisliu commented 1 year ago

4bit quantization for inference and finetuning.