Open 1105135335 opened 3 weeks ago
初学者,还请见谅
Hi, can you try the solution here: https://github.com/huggingface/accelerate/issues/2726?
试了试,解决了。非常感谢。
所以,这个没问题,我可以理解为是因为没有GPU的原因吗?
第一次出现这个问题,修改的部分如下:
# model = AutoModelForCausalLM.from_pretrained(
# model_dir, trust_remote_code=trust_remote_code, device_map='auto'
# )
model = transformers.pipeline(
"text-generation",
model=model_dir,
model_kwargs={"torch_dtype": torch.bfloat16},
device_map="auto",
)
结束程序并未做任何修改,又再一次重新在conda虚拟环境的控制台输入python trans_web_demo.py运行程序,和上次运行同样的步骤,同样的环境,同样的代码,又报了如下这个错误:
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<?, ?it/s]
Traceback (most recent call last):
File "G:\project\ht\240815\LongWriter\trans_web_demo.py", line 68, in <module>
model, tokenizer = load_model_and_tokenizer(MODEL_PATH, trust_remote_code=True)
File "G:\project\ht\240815\LongWriter\trans_web_demo.py", line 55, in load_model_and_tokenizer
model = transformers.pipeline(
File "F:\Tool\Anaconda3\envs\ht240815\lib\site-packages\transformers\pipelines\__init__.py", line 895, in pipeline
framework, model = infer_framework_load_model(
File "F:\Tool\Anaconda3\envs\ht240815\lib\site-packages\transformers\pipelines\base.py", line 299, in infer_framework_load_model
raise ValueError(
ValueError: Could not load model G:\project\ht\240815\LongWriter\THUDM\LongWriter-llama3.1-8b with any of the following classes: (<class 'transformers.models.auto.modeling_auto.AutoModelForCausalLM'>, <class 'transformers.models.llama.modeling_llama.LlamaForCausalLM'>). See the original errors:
while loading with AutoModelForCausalLM, an error is thrown:
Traceback (most recent call last):
File "F:\Tool\Anaconda3\envs\ht240815\lib\site-packages\transformers\pipelines\base.py", line 286, in infer_framework_load_model
model = model_class.from_pretrained(model, **kwargs)
File "F:\Tool\Anaconda3\envs\ht240815\lib\site-packages\transformers\models\auto\auto_factory.py", line 564, in from_pretrained
return model_class.from_pretrained(
File "F:\Tool\Anaconda3\envs\ht240815\lib\site-packages\transformers\modeling_utils.py", line 4015, in from_pretrained
dispatch_model(model, **device_map_kwargs)
File "F:\Tool\Anaconda3\envs\ht240815\lib\site-packages\accelerate\big_modeling.py", line 496, in dispatch_model
raise ValueError(
ValueError: You are trying to offload the whole model to the disk. Please use the `disk_offload` function instead.
while loading with LlamaForCausalLM, an error is thrown:
Traceback (most recent call last):
File "F:\Tool\Anaconda3\envs\ht240815\lib\site-packages\transformers\pipelines\base.py", line 286, in infer_framework_load_model
model = model_class.from_pretrained(model, **kwargs)
File "F:\Tool\Anaconda3\envs\ht240815\lib\site-packages\transformers\modeling_utils.py", line 4015, in from_pretrained
dispatch_model(model, **device_map_kwargs)
File "F:\Tool\Anaconda3\envs\ht240815\lib\site-packages\accelerate\big_modeling.py", line 496, in dispatch_model
raise ValueError(
ValueError: You are trying to offload the whole model to the disk. Please use the `disk_offload` function instead.
你好,我们更新了模型的modeling_chatglm.py以及github中的trans_web_demo.py,请试一下更新之后是否还有问题?
非常真诚的表达感谢,我是新手,还望见谅,提的问题可能会比较简单且奇怪,实在是抱歉。我会再次尝试,并及时反馈。
你好,我们更新了模型的modeling_chatglm.py以及github中的trans_web_demo.py,请试一下更新之后是否还有问题?
请问怎么解决的,我也是这个问题,有没有最后能run的码,第一次玩这个,求带
(ht240815) PS G:\project\ht\240815\LongWriter> python .\trans_web_demo.py Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<?, ?it/s] Traceback (most recent call last): File "G:\project\ht\240815\LongWriter\trans_web_demo.py", line 59, in <module> model, tokenizer = load_model_and_tokenizer(MODEL_PATH, trust_remote_code=True) File "G:\project\ht\240815\LongWriter\trans_web_demo.py", line 49, in load_model_and_tokenizer model = AutoModelForCausalLM.from_pretrained( File "F:\Tool\Anaconda3\envs\ht240815\lib\site-packages\transformers\models\auto\auto_factory.py", line 564, in from_pretrained return model_class.from_pretrained( File "F:\Tool\Anaconda3\envs\ht240815\lib\site-packages\transformers\modeling_utils.py", line 4015, in from_pretrained dispatch_model(model, **device_map_kwargs) File "F:\Tool\Anaconda3\envs\ht240815\lib\site-packages\accelerate\big_modeling.py", line 496, in dispatch_model raise ValueError( ValueError: You are trying to offload the whole model to the disk. Please use the
disk_offloadfunction instead.
昨晚还一切正常,运行起来了,同样的代码,啥也没动,今个一直报这个。下面是transformers的版本:
> pip show transformers Name: transformers Version: 4.44.0 Summary: State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow Home-page: https://github.com/huggingface/transformers Author: The Hugging Face team (past and future) with the help of all our contributors (https://github.com/huggingface/transformers/graphs/contributors) Author-email: transformers@huggingface.co License: Apache 2.0 License Location: f:\tool\anaconda3\envs\ht240815\lib\site-packages Requires: filelock, huggingface-hub, numpy, packaging, pyyaml, regex, requests, safetensors, tokenizers, tqdm Required-by: peft