The Orion-14B-Chat-Int4 model miss the chat method

OrionStarAI / Orion

Orion-14B is a family of models includes a 14B foundation LLM, and a series of models: a chat model, a long context model, a quantized model, a RAG fine-tuned model, and an Agent fine-tuned model. Orion-14B 系列模型包括一个具有140亿参数的多语言基座大模型以及一系列相关的衍生模型，包括对话模型，长文本模型，量化模型，RAG微调模型，Agent微调模型等。

Apache License 2.0

785 stars 57 forks source link

The Orion-14B-Chat-Int4 model miss the chat method #7

Open Xls1994 opened 9 months ago

Xls1994 commented 9 months ago

When I use the Orion-14B-Chat-Int4 model, it meets the following error.

response = model.chat(tokenizer, messages) File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1695, in __getattr__ raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'") AttributeError: 'OrionForCausalLM' object has no attribute 'chat'

a154377713 commented 9 months ago

I have the same problem: "AttributeError: 'OrionForCausalLM' object has no attribute 'chat'"

Xls1994 commented 9 months ago

When I updated the code, I got another error，the RuntimeError: CUDA error: no kernel image is available for execution on the device. Anyone helps me?

cuda version 11.8 awq version：0.18+cu118

KIM7AZEN commented 9 months ago

When I updated the code, I got another error，the RuntimeError: CUDA error: no kernel image is available for execution on the device. Anyone helps me?

cuda version 11.8 awq version：0.18+cu118

I have the same err . do you solve this?