加载模型报错 - Githubissues

FreedomIntelligence / HuatuoGPT-II

HuatuoGPT2, One-stage Training for Medical Adaption of LLMs. (An Open Medical GPT)

360 stars 59 forks source link

加载模型报错 #11

Closed iris-qq closed 10 months ago

iris-qq commented 10 months ago

File "/Users/iris/PycharmProjects/sino-cdss-customer/llm/huaTuo2_LLM.py", line 31, in load_model_tokenizer
    model = AutoModelForCausalLM.from_pretrained(self.model_path, device_map="auto", torch_dtype="auto", trust_remote_code=True)
.......

importlib.metadata.PackageNotFoundError: No package metadata was found for auto-gptq

系统版本信息

macOS Sonoma 14.2.1(23C71)
python 3.11
transformers==4.36.2

iris-qq commented 10 months ago

model是huggingface下载的 HuatuoGPT2-7B-4bits

iris-qq commented 10 months ago

确认了，mac m1芯片不支持量化版本