[Feature Request] Enable offline Speech-to-Text and Text-to-Speech

jovotech / jovo-framework

🔈 The React for Voice and Chat: Build Apps for Alexa, Messenger, Instagram, the Web, and more

https://www.jovo.tech

Apache License 2.0

1.68k stars 310 forks source link

[Feature Request] Enable offline Speech-to-Text and Text-to-Speech #1237

Open JRMeyer opened 2 years ago

JRMeyer commented 2 years ago

👋 hi there!

I'm submitting a...

[ ] Bug report
[x] Feature request
[ ] Documentation issue or request
[ ] Other... Please describe:

Expected Behavior

Would be great to be able to test and debug a voice bot without an internet connection. Offline STT and TTS (from @coqui-ai) would make this possible using the existing UX from the new jovo debugger

Current Behavior

Currently there's no offline STT or TTS

jankoenig commented 2 years ago

Hi there. Thank you.

This is not on our immediate roadmap, but would be a great community contribution.

Coqui STT could be implemented as Jovo ASR integration.

JRMeyer commented 2 years ago

Hi @jankoenig -- just looked into the integration with Lex, and it would be considerably different with Coqui because the user would have their own server running. For example, the user might be running a simple server on their local desktop or they might have spun up a server on their AWS cloud, and using endpoints there. In either case, the API syntax and integration would be identical, but there would be an expectation that the user spins up the server themselves. Not too difficult, but I'm not sure if that's something the Jovo crowd would be interested in.

I think the biggest value add for Jovo users would be to be able to test out their voicebots locally, without having an ASR backend running on one of the providers (like Lex).

Thoughts?

jankoenig commented 2 years ago

This could work similar to our Snips NLU integration where people also have to run their own servers.

An integration like this would also be useful for our web starters:

JRMeyer commented 2 years ago

Yeah, I think a general setup mirroring the Snips approach would work nicely. You know of anyone in your community who might like to hack on this? We're happy to offer support/guidance for using the Coqui tools.

rubenaeg commented 2 years ago

I think I could give this a spin :)

rmtuckerphx commented 2 years ago

@JRMeyer Are there any developer docs on the Coqui APIs for STT and TTS using Node.js or REST?