Closed chris-tng closed 3 months ago
Seems like 192 = 63 4, and 256 = 64 4. Judging from the error codes, it seems like there was a missing <image>
placeholder in the text. You probably need to check your input data for debugging, ensuring that the number of input <image>
tokens in the text has the same number of the images.
Thank you for your reply @jdf-prog . It's likely the error. I have resolved it by using cpu_offload. I think truncation actually cuts off
Sorry for my newbie question as I'm new to VLM. Do you have any rule of thumb to estimate VRAM usage for idefics2-8b with multiple images (I'm using 4 for each prompt) ?
Also, I see that max_image_size='(1080,1920)'
, do I need to resize images of resolution is larger than this?
I'm working on fine-tuning Idefics2 with multiple images in instruction I follow this script for full fine-tuning: https://github.com/TIGER-AI-Lab/Mantis/blob/89d34077bd87b66eaadc13117add553e3a3d4c0b/mantis/train/scripts/train_idefics2_full.sh
Here is the command
Error i got is
Any suggestions how to fix it?
Thanks in advance