search

sanchit-gandhi / whisper-jax

JAX implementation of OpenAI's Whisper model for up to 70x speed-up on TPU.
Apache License 2.0
4.38k stars 369 forks source link

Can I use forced alignment? #74

Open NielsVandenEynde opened 1 year ago

NielsVandenEynde commented 1 year ago

I'm creating a dataset and need each segment to be a sentence, no cutoffs. I can do this with forced alignment with whisperx, is it possible to somehow get it working with this version? The speed would be really nice.

sanchit-gandhi commented 1 year ago

Probably you can replace OpenAI's Whisper model with Whisper JAX in the Whisper-X repo? E.g. you want to replace this line: https://github.com/m-bain/whisperX/blob/b6665230042406c7c76d91b8a7c46dc88c2a2bd0/whisperx/transcribe.py#L165 With Whisper JAX inference

Jrcordal commented 1 month ago

I'm creating a dataset and need each segment to be a sentence, no cutoffs. I can do this with forced alignment with whisperx, is it possible to somehow get it working with this version? The speed would be really nice.

did it work at the end?