Whisper support for Speech to Text and Elevenlabs support for Text to Speech

livekit-examples / kitt

Talk to ChatGPT in real time using LiveKit

https://kitt.livekit.io/

Apache License 2.0

214 stars 62 forks source link

Whisper support for Speech to Text and Elevenlabs support for Text to Speech #16

Open maddyonline opened 8 months ago

maddyonline commented 8 months ago

This is a great package. I am wondering if we want to implement a custom Transcriber (say based on OpenAI's whisper) how would we go about doing it? Any code snippets (even if incomplete) would be useful. In particular, how to convert incoming packets compatible with Whisper?

Likewise if we want to use a different Synthesizer like ElevenLabs what is a good place to start.

I would be happy to contribute these additions to the package if I can get some help on getting started.

lmsutools commented 7 months ago

I suscribe to this idea. Also, ElevenLabs with stream option to receive faster audios.