search

yeyupiaoling / Whisper-Finetune

Fine-tune the Whisper speech recognition model to support training without timestamp data, training with timestamp data, and training without speech data. Accelerate inference and support Web deployment, Windows desktop deployment, and Android deployment
Apache License 2.0
867 stars 143 forks source link

Token length limit #96

Open rose-jinyang opened 1 week ago

rose-jinyang commented 1 week ago

Hello How are you? Thanks for contributing to this project. I am going to fine-tune Whisper model for Indian Telugu language on google/fleurs dataset.

torchrun --nproc_per_node=2 finetune.py --base_model=openai/whisper-large-v2 --language=None

But while training, I met the following issue.

File "/opt/conda/lib/python3.10/site-packages/transformers/models/whisper/modeling_whisper.py", line 1757, in forward raise ValueError( ValueError: Labels' sequence length 495 cannot exceed the maximum allowed length of 448 tokens.

What do you think about possible reasons?

yeyupiaoling commented 1 week ago

@rose-jinyang whisper has a limit on the input text, the length of each audio text cannot exceed 448 tokens, so you need to filter the data, this project only limits the length of the audio.