tastelikefeet
commented
2 months ago I cannot reproduce the problem
I trained mini-internvl-chat-4b-v1_5 and merged lora into the original model, then infered with
swift infer --ckpt_dir xxx/checkpoint-1-mergedBut everything seemed Ok.
babla9
Jintao-Huang
Describe the bug What the bug is, and how to reproduce, better with screenshots(描述bug以及复现过程，最好有截图) When running inference on a fine-tuned model, I get the following error:
File "/home/ubuntu/.cache/huggingface/modules/transformers_modules/checkpoint-3093-merged/modeling_phi3.py", line 1110, in forward position_ids = position_ids.view(-1, seq_length).long() RuntimeError:shape '[-1, 0]' is invalid for input of size 1325`Stack trace:
[INFO:swift] Please enter the conversation content first, followed by the path to the multimedia file. <<< describe this image Input a media path or URL <<< http://modelscope-open.oss-cn-hangzhou.aliyuncs.com/images/cat.png Theseen_tokensattribute is deprecated and will be removed in v4.41. Use thecache_positionmodel input instead. Exception in thread Thread-2 (generate): Traceback (most recent call last): File "/opt/conda/envs/swift-env/lib/python3.10/threading.py", line 1016, in _bootstrap_inner self.run() File "/opt/conda/envs/swift-env/lib/python3.10/threading.py", line 953, in run self._target(*self._args, **self._kwargs) File "/home/ubuntu/swift/swift/llm/utils/model.py", line 3600, in _new_generate return generate(*args, **kwargs) File "/opt/conda/envs/swift-env/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/home/ubuntu/.cache/huggingface/modules/transformers_modules/checkpoint-3093-merged/modeling_internvl_chat.py", line 331, in generate outputs = self.language_model.generate( File "/opt/conda/envs/swift-env/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/opt/conda/envs/swift-env/lib/python3.10/site-packages/transformers/generation/utils.py", line 1914, in generate result = self._sample( File "/opt/conda/envs/swift-env/lib/python3.10/site-packages/transformers/generation/utils.py", line 2651, in _sample outputs = self( File "/opt/conda/envs/swift-env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/opt/conda/envs/swift-env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl return forward_call(*args, **kwargs) File "/home/ubuntu/swift/swift/llm/utils/model.py", line 3623, in _new_forward output = old_forward(*args, **kwargs) File "/home/ubuntu/.cache/huggingface/modules/transformers_modules/checkpoint-3093-merged/modeling_phi3.py", line 1281, in forward outputs = self.model( File "/opt/conda/envs/swift-env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/opt/conda/envs/swift-env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl return forward_call(*args, **kwargs) File "/home/ubuntu/.cache/huggingface/modules/transformers_modules/checkpoint-3093-merged/modeling_phi3.py", line 1110, in forward position_ids = position_ids.view(-1, seq_length).long() RuntimeError: shape '[-1, 0]' is invalid for input of size 1325Your hardware and system info Write your system info like CUDA version/system/GPU/torch version here(在这里给出硬件信息和系统信息，如CUDA版本，系统，GPU型号和torch版本等)
Additional context Add any other context about the problem here(在这里补充其他信息)