huggingface / community-events

Place where folks can contribute to 🤗 community events
400 stars 100 forks source link

How to prepare audio dataset for whisper fine-tuning with timestamps? #200

Open omarabb315 opened 9 months ago

omarabb315 commented 9 months ago

I am trying to prepare a dataset for whisper fine-tuning , and I have a lot of small segment clip , most of them less than 6 seconds, I read the paper, but didn’t understand this paragraph:

“ When a final transcript segment is only partially included in the current 30- second audio chunk, we predict only its start time token for the segment when in timestamp mode, to indicate that the subsequent decoding should be performed on an audio window aligned with that time, otherwise we truncate the audio to not include the segment”

So when should I add the final segment if it is partially included in the current 30-second chunk, and when should I truncate the chunk without it, and if I added it how to extract only relevant transcription?

To make it clear:

|           window           |           window           |
|segment|-----segment---|--segment--|

assume that every window is 30 seconds, how to get the correct relevant transcription of the partially included segments? Anyone could help?