search

LlamaFamily / Llama-Chinese

Llama中文社区，Llama3在线体验和微调模型已开放，实时汇总最新Llama3学习资料，已将所有代码更新适配Llama3，构建最好的中文Llama大模型，完全开源可商用
https://llama.family
14k stars 1.26k forks source link

运行quikstart进行模型推理报错 #337

Open sunt40 opened 6 months ago

sunt40 commented 6 months ago

This modeling file requires the following packages that were not found in your environment: flash_attn. Run pip install flash_attn

sunt40 commented 6 months ago

安装不了这个包

wangzilong1997 commented 6 months ago

。。。。mac m1 一样这个问题

shams2023 commented 6 months ago

如何解决？

wytitan commented 6 months ago

trust_remote_code=True, use_flash_attention_2=True 我把这两个都移除了可以加载了，参考https://huggingface.co/microsoft/phi-1_5/discussions/72

我在纯CPU环境，改成了AutoModelForCausalLM.from_pretrained(model_name_or_path, device_map=device_map, torch_dtype=torch.float16)