Open uprokevin opened 11 months ago
Thanks for the question. I do not think it would be hard to support Mistral. We will take a look when there is bandwidth.
You may also try modifying the code to support Mistral following the logistics here: https://github.com/mit-han-lab/llm-awq/blob/main/awq/quantize/pre_quant.py
Any update on this?
Any update?
Checking the constraints to add Mistral 7B in the list of models.
It seems it has been benchmarked with auto-awq :
https://github.com/casper-hansen/AutoAWQ