Closed xing0047 closed 4 months ago
Thanks for your appreciation!
Have you tried to reproduce our POPE results? Will this bug still exist?
Thanks for your kind reply.
We can reproduce POPE results. There are no "size mismatch" bugs reported when we reproduce POPE.
Hi,
Could you please provide the image id that triggered this error? I have checked and rerun the code locally and haven't found any problems.
This problem might be the Out-of-Memory error triggered at https://github.com/shikiw/OPERA/blob/b83bc6b264a2d7bd4a6163fc21f9bb806d443a0b/transformers-4.29.2/src/transformers/generation/utils.py#L3515-L3520 and then it jumps out "try" to "except" without running the code at (https://github.com/shikiw/OPERA/blob/b83bc6b264a2d7bd4a6163fc21f9bb806d443a0b/transformers-4.29.2/src/transformers/generation/utils.py#L3534-L3552 Thus, model_kwargs["past_key_values"] can not be rolled back successfully.
Solution 1: Try it on the GPU with larger memory
Solution 2: Try beam=2
and num_attn_candidates=2
.
Hi shikiw,
Thanks for replying to this issue.
It appears that problem is not underlying in image inputs. As suggested, I will try these two solutions first and see if there is any problem.
same bug.
do you have some process on solving this bug? @xing0047
It seems that attn_previous
and output.attentions
dimension 2(query dimension) are mismatch. Maybe the rollback causes the bug, I am not sure.
Thanks for your great work and open-sourcing your codes!
I am working on reproducing CHAIR results for OPERA (which are presented in Table 2 and 3). There is a SizeMismatch error when running
python chair_eval.py
over single 4090 GPU. May I ask if there is something wrong with my evaluation scripts? For reference, both evaluation scripts and reported errors are presented as follows (have tried bothmax_new_tokens=512
andmax_new_tokens=64
and encounter the same issue).evaluation script
reported error (happens after a few steps)