search

Alpha-VLLM / LLaMA2-Accessory

An Open-source Toolkit for LLM Development
https://llama2-accessory.readthedocs.io/
Other
2.68k stars 170 forks source link

update Quantization doc #59

Closed kriskrisliu closed 1 year ago

kriskrisliu commented 1 year ago
  1. Introduce QPEFT: QNormBias and QNormBiasLoRA
  2. Fine-tuning scripts
  3. Comparing to bf16 in training/inference (shown in tables)
  4. Inference scripts