search

AutoGPTQ / AutoGPTQ

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.
MIT License
4.37k stars 467 forks source link

[FEATURE] Cohere integration? #612

Closed DavidePaglieri closed 1 month ago

DavidePaglieri commented 6 months ago

Hi, would it be possible to officially support the new Cohere model introduced in transformers=4.39? Thanks!

egortolmachev commented 6 months ago

Ready: https://github.com/AutoGPTQ/AutoGPTQ/pull/631