Closed tctrautman closed 3 months ago
Hey! Indeed Flash-attention seems to be broken in the last release caused by https://github.com/huggingface/transformers/pull/31629. I located the reason and will work on fix, in the meanwhile you can downgrade transformers version to at most v.4.42.4, and try generating again :)
Thank you, @zucchini-nlp!
System Info
Who can help?
@zucchini-nlp
Information
Tasks
examples
folder (such as GLUE/SQuAD, ...)Reproduction
The below script is the same as the one that is included on the Idefics2 blog post, with three additional lines added within
AutoModelForVision2Seq.from_pretrained
, with comments to note the new lines.When this block of code is run, it will yield the below error.
Expected behavior
I'd expect the above script to generate without error (a similar one did earlier this week, which now yields the same error.)
I believe one of these two issues might be related to this issue: