Closed hpy-42 closed 2 months ago
Thank you for pointing this issue. It is a bug and we will fix it soon. In our demo and evaluation, we turn off image cut, so it temporarily does not affect the performance of the model in most scenarios.
We fixed this issue and updated the code in Hugging Face and ModelScope. However, we found that because we never trained the model with multi-image input when enabling the image cut, the performance is suboptimal. We will improve this weakness in the next model release.
https://huggingface.co/mPLUG/mPLUG-Owl3-7B-240728/blob/main/processing_mplugowl3.py#L232
When
self.image_processor.add_global
set to True, i thinkimage_token_ptr
should += 1 one more time during the loop...