microsoft / Olive

Olive: Simplify ML Model Finetuning, Conversion, Quantization, and Optimization for CPUs, GPUs and NPUs.
https://microsoft.github.io/Olive/
MIT License
1.53k stars 163 forks source link

Make certain gptq options customizable via model specific mapping #1363

Closed shaahji closed 1 week ago

shaahji commented 2 weeks ago

Make certain gptq options customizable via model specific mapping

This is to avoid hardcoding these paramters in config files for models that aren't (like phi3) yet officially supported by auto-gptq.

Checklist before requesting a review

(Optional) Issue link