how about the qulity of the batched faster-whisper？

KoljaB / RealtimeSTT

A robust, efficient, low-latency speech-to-text library with advanced voice activity detection, wake word activation and instant transcription.

MIT License

2.09k stars 190 forks source link

how about the qulity of the batched faster-whisper？ #90

Open jacobtang opened 3 months ago

jacobtang commented 3 months ago

from faster_whisper import WhisperModel, BatchedInferencePipeline

model = WhisperModel("medium", device="cuda", compute_type="float16") batched_model = BatchedInferencePipeline(model=model)

Batched version improves the speed upto 10-12x compared to openAI implementation and 3-4x compared to the sequential faster_whisper version. It works by transcribing semantically meaningful audio chunks as batches leading to faster inference.

Will reatimeSTT support the batched version? Thanks!

KoljaB commented 3 months ago

Will look into that.