Fix deepseekv2 quant bug and support chat calib data

ModelTC / llmc

[EMNLP 2024 Industry Track] This is the official PyTorch implementation of "LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit".

https://arxiv.org/abs/2405.06001

Apache License 2.0

326 stars 34 forks source link

Fix deepseekv2 quant bug and support chat calib data #182

Closed llmc-reviewer closed 2 weeks ago