Closed pranay-ar closed 3 weeks ago
Hello! You don't get this error in v4.41.0?
Moondream is not maintained by us unfortunately, but we'd be happy to see what change in transformers
resulted in the break to see how we could fix it.
Yes, the code works perfectly fine until 4.41.2. It only breaks from the 4.42.0+ releases!
Yes, the code works perfectly fine until 4.41.2. It only breaks from the 4.42.0+ releases!
Hi, @pranay-ar curious if you find any fix on this?
Hi @hypernovas as per the author of Moondream, there is a new version coming up soon, so I thought of waiting for that one and reverted back to 4.41.2 for my use-case.
@pranay-ar Thanks! I am actually getting this problem with lower transformers version,
accelerate==0.25.0 huggingface-hub==0.20.1 Pillow==10.1.0 torch==2.1.2 torchvision==0.16.2 transformers==4.36.2 einops==0.7.0 gradio==4.15.0 flash-attn==2.5.8
Do you mind share what are the lib versions work for you? Really appreciate that. I am thinking one of their dependencies or other libs got upgraded. And maybe you also don't wanna to upgrade any libs ;)
Hey @hypernovas sorry for the late reply, I found the fix and it was merged into main branch today, can you try again and let me know if you're still facing that issue!
These are my library versions for your reference:
accelerate==0.31.0 huggingface-hub==0.23.3 Pillow==10.3.0 torch==2.2.0 torchvision==0.17.0 transformers==4.42.4 einops==0.8.0
Hello, fyi I have a similar problem with a similar VLM I made using phi 3 mini. Using input_embeds with model.generate I have the error (I dont have it with 4.41.1) : File "/opt/conda/envs/ptca/lib/python3.10/site-packages/transformers/generation/utils.py", line 1914, in generate result = self._sample( File "/opt/conda/envs/ptca/lib/python3.10/site-packages/transformers/generation/utils.py", line 2651, in _sample outputs = self( File "/opt/conda/envs/ptca/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, kwargs) File "/opt/conda/envs/ptca/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl return forward_call(*args, *kwargs) File "/root/.cache/huggingface/modules/transformers_modules/microsoft/Phi-3-mini-4k-instruct/c1358f8a35e6d2af81890deffbbfa575b978c62f/modeling_phi3.py", line 1243, in forward outputs = self.model( File "/opt/conda/envs/ptca/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl return self._call_impl(args, kwargs) File "/opt/conda/envs/ptca/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl return forward_call(*args, **kwargs) File "/root/.cache/huggingface/modules/transformers_modules/microsoft/Phi-3-mini-4k-instruct/c1358f8a35e6d2af81890deffbbfa575b978c62f/modeling_phi3.py", line 1072, in forward position_ids = position_ids.view(-1, seq_length).long() RuntimeError: shape '[-1, 0]' is invalid for input of size 50
System Info
transformers
version: 4.42.3Who can help?
@amyeroberts
Information
Tasks
examples
folder (such as GLUE/SQuAD, ...)Reproduction
Expected behavior