trans_cli_vision_demo.py 出现报错？

[michaelwithu]

环境， transformers 版本，4.40.0 torch版本，2.3.0+cu121 其中，加载的模型，是通过modelscope下载的。 下面是加载的代码， MODEL_PATH = os.environ.get('MODEL_PATH', '/root/.cache/modelscope/hub/ZhipuAI/glm-4v-9b')

tokenizer = AutoTokenizer.from_pretrained( MODEL_PATH, trust_remote_code=True, encode_special_tokens=True ) model = AutoModel.from_pretrained( MODEL_PATH, trust_remote_code=True, device_map="auto", torch_dtype=torch.bfloat16 ).eval()

(glm4-basic) glm4# python basic_demo/trans_cli_vision_demo.py clear Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained. Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 15/15 [00:01<00:00, 9.36it/s] 2.3.0+cu121

Welcome to the GLM-4-9B CLI chat. Type your messages below. Image Path:/home/mic/code/github/glm4/data/jxxq_6.jpeg

You: what can you see?

报错信息， GLM-4V:Exception in thread Thread-2 (generate): Traceback (most recent call last): File "/root/anaconda3/envs/glm4-basic/lib/python3.10/threading.py", line 1016, in _bootstrap_inner self.run() File "/root/anaconda3/envs/glm4-basic/lib/python3.10/threading.py", line 953, in run self._target(*self._args, self._kwargs) File "/root/anaconda3/envs/glm4-basic/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, *kwargs) File "/root/anaconda3/envs/glm4-basic/lib/python3.10/site-packages/transformers/generation/utils.py", line 1622, in generate result = self._sample( File "/root/anaconda3/envs/glm4-basic/lib/python3.10/site-packages/transformers/generation/utils.py", line 2791, in _sample outputs = self( File "/root/anaconda3/envs/glm4-basic/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/root/anaconda3/envs/glm4-basic/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl return forward_call(*args, kwargs) File "/root/anaconda3/envs/glm4-basic/lib/python3.10/site-packages/accelerate/hooks.py", line 166, in new_forward output = module._old_forward(*args, *kwargs) File "/root/.cache/huggingface/modules/transformers_modules/glm-4v-9b/modeling_chatglm.py", line 1012, in forward transformer_outputs = self.transformer( File "/root/anaconda3/envs/glm4-basic/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/root/anaconda3/envs/glm4-basic/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl return forward_call(*args, kwargs) File "/root/.cache/huggingface/modules/transformers_modules/glm-4v-9b/modeling_chatglm.py", line 849, in forward images_features = self.vision(images) File "/root/anaconda3/envs/glm4-basic/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl return self._call_impl(*args, *kwargs) File "/root/anaconda3/envs/glm4-basic/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl return forward_call(args, kwargs) File "/root/.cache/huggingface/modules/transformers_modules/glm-4v-9b/visual.py", line 157, in forward x = self.transformer(x) File "/root/anaconda3/envs/glm4-basic/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl return self._call_impl(*args, kwargs) File "/root/anaconda3/envs/glm4-basic/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl return forward_call(*args, *kwargs) File "/root/.cache/huggingface/modules/transformers_modules/glm-4v-9b/visual.py", line 120, in forward hidden_states = layer_module(hidden_states) File "/root/anaconda3/envs/glm4-basic/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/root/anaconda3/envs/glm4-basic/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl return forward_call(*args, kwargs) File "/root/anaconda3/envs/glm4-basic/lib/python3.10/site-packages/accelerate/hooks.py", line 166, in new_forward output = module._old_forward(*args, *kwargs) File "/root/.cache/huggingface/modules/transformers_modules/glm-4v-9b/visual.py", line 105, in forward attention_output = self.input_layernorm(self.attention(attention_input)) File "/root/anaconda3/envs/glm4-basic/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/root/anaconda3/envs/glm4-basic/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl return forward_call(*args, *kwargs) File "/root/anaconda3/envs/glm4-basic/lib/python3.10/site-packages/accelerate/hooks.py", line 166, in new_forward output = module._old_forward(args, **kwargs) File "/root/.cache/huggingface/modules/transformers_modules/glm-4v-9b/visual.py", line 69, in forward output = self.dense(out.transpose(1, 2).view(B, L, -1)) RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.

根据多gpu的issue，重新下载了新的文件，上面报错解决了，但是还是出现多gpu 的报错， RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:2 and cuda:3!

[orderer0001]

我也是同样的问题 多卡 只要不是自动分配 就有问题 因为牵涉到其他考虑 所以无法自动分配

[CX26-CX]

有解决方法吗