Closed hieunguyenquoc closed 1 month ago
Gentle ping @sanchit-gandhi
I believe the problem lies here:
The check for if "encoder_outputs" in model_kwargs
on line 128
should be above the check for if assistant_model.config.is_encoder_decoder
on line 121
, because otherwise the outputs of the main model's encoder are fed in as inputs to the encoder of the assistant model, when they should be just used as inputs for the assistant decoder.
Happy to submit a PR for this.
Thanks for reporting @hieunguyenquoc! A PR would be most welcome @jdvin if you have the bandwidth, otherwise cc @kamilakesbi if you could take a look
This issue has been solved with PR #30637 :)
System Info
transformers==4.39.1 python==3.8.17 torch==2.0.1+cpu
Who can help?
@sanchit-gandhi
Information
Tasks
examples
folder (such as GLUE/SQuAD, ...)Reproduction
Expected behavior
I have tried Speculative Decoding on two version of PhoWhisper (Whisper finetuned version) followed by this post [https://huggingface.co/blog/whisper-speculative-decoding]. I have this error : ValueError: Whisper expects the mel input features to be of length 3000, but found 1500. Make sure to pad the input mel features to 3000. Could you help me ? Thank you @sanchit-gandhi