support GPTNEOX model - Githubissues

mit-han-lab / smoothquant

[ICML 2023] SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models

https://arxiv.org/abs/2211.10438

MIT License

1.1k stars 127 forks source link

support GPTNEOX model #40

Open amazingkmy opened 1 year ago

amazingkmy commented 1 year ago

Hi I am working on quantization using the gptneox model. During the quantization process, I got the following message.

"You are using a model of type gpt_neox to instantiate a model of type opt. This is not supported for all configurations of models and can yield errors."

Is GPTNEOX conversion not possible?