search

alphacep / vosk-server

WebSocket, gRPC and WebRTC speech recognition server based on Vosk and Kaldi libraries
Apache License 2.0
882 stars 243 forks source link

WebRTC over HTTPS #200

Closed yne closed 1 year ago

yne commented 1 year ago

Context

I'm trying to use the WebRTC example over an HTTPS connection on a separate machine.

Issue

The secure demo page display correctly and the "start button" trigger the mic request but it stay stuck at the "connecting" stage even if the POST /offer reply is okay.

The same docker command on a my local machine (with custom certif) worked.

hypothesis

WebRTC need to open a dynamic for the voice streaming, which is not compatible with the docker static port configuration.

Steps to reproduce

nshmyrev commented 1 year ago

Yes, you need to forward/open other webrtc ports on docker at least these:

30000:60000/udp ALLOW Anywhere
1544,5060,5090,8060/tcp ALLOW Anywhere
1544,5060,5090,8060/udp ALLOW Anywhere

It is probably easier to try without docker first and disable firewall

yne commented 1 year ago

Thanks for the confirmations,

Is there any shorter port range than 30000-60000 ?

Every docker port is handled by a docker-proxy process, so having a -p 30000-60000 in my docker run argument would spawn ~30.000 processes which my kernel don't seems to appreciate past 4900 PID

I tried with docker run -v $PWD/cert:/cert -it -p 30000:30000 -p 80:80 -p 443:443 -p 1544:1544 -p 5060:5060 -p 5090:5090 -p 8060:8060 alphacep/kaldi-en:latest /bin/bash without success tho

Anecdotally, I see those warning (but it's probably nothing important)

/usr/local/lib/python3.9/dist-packages/aiortc/rtcdtlstransport.py:211: CryptographyDeprecationWarning: This version of cryptography contains a temporary pyOpenSSL fallback path. Upgrade pyOpenSSL now.
  _openssl_assert(lib.SSL_CTX_use_certificate(ctx, self._cert._x509) == 1)  # type: ignore
/usr/local/lib/python3.9/dist-packages/aiortc/rtcdtlstransport.py:186: CryptographyDeprecationWarning: This version of cryptography contains a temporary pyOpenSSL fallback path. Upgrade pyOpenSSL now.
  value=certificate_digest(self._cert._x509),  # type: ignore
nshmyrev commented 1 year ago

yes, it is a common problem with rtp port ranges, see

https://www.engagespark.com/blog/rtp-port-ranges-for-freeswitch-in-docker

telephony guys don't like docker.

for ssl it is harmless, but we recommend to use nginx frontend for ssl actually and run the server without ssl

yne commented 1 year ago

Yes, the original setup use Nginx but to find the culprit I tried a more standalone approach.

In the end it was docker (and sadly --network=host did not worked for me).

Thanks for your lights

nothingcomeseasy commented 1 year ago

hey guys. i'm loving vosk-server!

Followed the instrunctions of WebRTC and vosk-server worked properly on localhost and in another machine on my LAN!

Now I'm wondering if is it possible to open it outside, for "the internet"? Could someone point me any direction?

Thanks!

yne commented 1 year ago

You'll need to configure your router to forward the ports listed here https://github.com/alphacep/vosk-server/issues/200#issuecomment-1254259061

nothingcomeseasy commented 1 year ago

@yne Yes. It's already working on my LAN environment, but let's say I already have a web host and a personal domain (.com).

I understand that apache and other "famous" servers have some kind of easy integration with SSL certificates.

I wonder if it the same applies to vosk-server? Can I run it under a real domain and make the microphone work? I'm having a hardtime to make it work on a digitalocean host. The pair domain/vosk-server runs good under the standard http 80 port (but in that case microphone isn't working) and I understand that it needs to run under https (443) to make the microphone work and will require a real SSL certificate.

yne commented 1 year ago

You can either use nginx as SSL proxy to your app (and keep the app in HTTP)

Or you simply use the app while providing the 2 needed env variables: https://github.com/alphacep/vosk-server/blob/master/webrtc/asr_server_webrtc.py#L22-L23

nothingcomeseasy commented 1 year ago

will try these!

thank you @yne :)

yne commented 1 year ago

no problem, I successfully switched a vosk project from webRTC to to webSocket. see: stt.js + nginx.conf

nothingcomeseasy commented 1 year ago

Could you elaborate? :)

What are the benefits of switching from webRTC to webSocket?

yne commented 1 year ago

In my case I needed an easy way to deploy and run my components (database, backend, speech-to-text, frontend ...). So I used a docker-compose which start my (postgress, postgrest, vosk, vue ...)

The issue is that webRTC use a wide range of ports, which is not compatible with my docker and VPS approach.

switching to websocket allowed me to keep a realtime approach (text appear as the user speak), while keeping the port range constrained (only 80+443 needed)

nothingcomeseasy commented 1 year ago

thanks for explaining @yne, sounds interesting! I'll read a little bit more about websockets, I'm not a dev, just an enthusiast prototyping a mvp with vosk-server.

I finally made webRTC to work with a real domain, following this thread, thanks for sharing your attemps, helped me a lot!

Do you think it's possible to make a scalable application using vosk-server through webRTC (aka handle multiple requests at the same time)?

yne commented 1 year ago

It would be an horrible design if vosk could only handle 1 request at a time