Loading checkpoint shards: 100%|███████████████████████████████████████████| 2/2 [02:01<00:00, 60.98s/it]
Traceback (most recent call last):
File "/home/kas/lx/code/MiniCPM-V/demo.py", line 20, in
answer = chatmodel.chat(inputs)
File "/home/kas/lx/code/MiniCPM-V/chat.py", line 197, in chat
return self.model.chat(input)
File "/home/kas/lx/code/MiniCPM-V/chat.py", line 153, in chat
answer, context, = self.model.chat(
File "/home/kas/.cache/huggingface/modules/transformers_modules/output_minicpmv2/modeling_minicpmv.py", line 355, in chat
res, vision_hidden_states = self.generate(
File "/home/kas/.cache/huggingface/modules/transformers_modules/output_minicpmv2/modeling_minicpmv.py", line 266, in generate
model_inputs = self._process_list(tokenizer, data_list, max_inp_length)
File "/home/kas/.cache/huggingface/modules/transformers_modules/output_minicpmv2/modeling_minicpmv.py", line 183, in _process_list
self._convert_to_tensors(tokenizer, data, max_inp_length)
File "/home/kas/.cache/huggingface/modules/transformers_modules/output_minicpmv2/modeling_minicpmv.py", line 163, in _convert_to_tensors
image_bound = torch.hstack(
RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 8 but got size 7 for tensor number 1 in the list.
训练sh脚本为:
!/bin/bash
GPUS_PER_NODE=8
NNODES=1
NODE_RANK=0
MASTER_ADDR=localhost
MASTER_PORT=6001
LLM_TYPE="minicpm" # if use openbmb/MiniCPM-V-2, please set LLM_TYPE=minicpm
是否已有关于该错误的issue或讨论? | Is there an existing issue / discussion for this?
该问题是否在FAQ中有解答? | Is there an existing answer for this in FAQ?
当前行为 | Current Behavior
微调后使用代码推理报错:
from chat import MiniCPMVChat, img2base64 import torch import json from peft import AutoPeftModelForCausalLM
model_path = "/home/kas/lx/code/MiniCPM-V/finetune/output/3d-100-new/output_minicpmv2" image_path = '/home/kas/lx/data/datasets/app_store_images/crop_norm_all/2368107_t0171411071ab005364.jpg'
torch.manual_seed(0)
chat_model = MiniCPMVChat(model_path)
im_64 = img2base64(image_path)
这里开头的\被md掉了,代码中是\\n定位寻城记....
question = "\n定位寻城记.\n以下是图片中可参考文本的坐标: '丁丁和爸爸'(346,615),'第二期'(123,727),'垃圾分类奇遇记'(382,855),'悦听'(365,083),'今晚,我们都是'(226,325),'新闻'(115,927),'视频'(508,927),'服务'(311,927),'寻城记'(135,551).\nAnswer:"
msgs = [{'role': 'user', 'content': question}]
inputs = {"image": im_64, "question": json.dumps(msgs)} answer = chat_model.chat(inputs) print(answer)
报错信息:
训练sh脚本为:
ds配置为:
训练数据格式为:
数据内图片分辨率为: 610 x 1344
关键库版本为: deepspeed 0.14.2 torch 2.3.0 transformers 4.40.2
期望行为 | Expected Behavior
No response
复现方法 | Steps To Reproduce
No response
运行环境 | Environment
备注 | Anything else?
No response