Open yuzeng0-0 opened 3 months ago
我们的实现方法加入了单独的image token标记图片位置,这在原模型中是没有的。您需要在模型的config.py
文件中添加"image_token_index":92544
感谢
File "/cache/zy/VL-RLHF/src/vlrlhf/models/InternLMXC2/init.py", line 87, in _merge_input_ids_with_image_features raise ValueError( ValueError: The input provided to the model are wrong. The number of image tokens is 2 while the number of image given to the model is 2. This prevents correct indexing and breaks batch generation.这个怎么解决
File "/cache/zy/VL-RLHF/src/vlrlhf/models/InternLMXC2/init.py", line 87, in _merge_input_ids_with_image_features raise ValueError( ValueError: The input provided to the model are wrong. The number of image tokens is 2 while the number of image given to the model is 2. This prevents correct indexing and breaks batch generation.这个怎么解决 if image_to_overwrite.sum() != image_features.shape[:-1].numel():条件判断raise ValueError
我们统一用<image>
作为image token,请检查下输入文本中image token的总数量与输入图片的总数量是否一致
我们统一用
<image>
作为image token,请检查下输入文本中image token的总数量与输入图片的总数量是否一致这个是没问题的。 我猜测是padding的问题,internXC2的pad_token_id是2,对应embedding为零向量,在执行final_embedding[image_to_overwrite] = image_features.contiguous().reshape(-1, embed_dim).to(target_device)的时候报错
方便展示一下internXC2模型的config.py文件吗
方便展示一下internXC2模型的config.py文件吗
可以参考我们放出的checkpoint: https://huggingface.co/TideDra/internlm-xcomposer2-vl-7b-DPO/blob/main/config.json
internXC2 的DPO代码还是跑不起来,您确定repo里是最新版本的代码吗?是不是repo是之前的版本导致有些bug没有修复
File "/home/ma-user/anaconda3/envs/dpo/lib/python3.10/site-packages/transformers/configuration_utils.py", line 264, in getattribute return super().getattribute(key) AttributeError: 'InternLMXcomposer2Config' object has no attribute 'image_token_index'