search

alumae / kaldi-gstreamer-server

Real-time full-duplex speech recognition server, based on the Kaldi toolkit and the GStreamer framwork.
BSD 2-Clause "Simplified" License
1.07k stars 341 forks source link

Permanent Worker-Master Websocket connection #128

Closed Thunderlbc closed 6 years ago

Thunderlbc commented 6 years ago

Hi~, I'm considering to serve speech recognition service with multiple workers (which have permanent ws connections with master) in the backend.

I've found that each time a client finishing pushing audio and sending eos, both the worker-master and master-client websocket connection will be closed sequentially.

Since there will be some small delays(https://github.com/alumae/kaldi-gstreamer-server/blob/master/kaldigstserver/worker.py#L367) when a new Worker-Master ws connection was established, this perhaps make the QPS( ~= number of available workers) lower than expected during some small interval( e.g. within 1s after client exit, this could fail the multiprocess pressure testing) .

Is there any possibilities that just keeping the worker-master ws connection, every time a client came in, master create a ws connection with it, plug this master-client connection onto random-chosen "permanent" worker-master connection, and start to decode?

And once finished(client sending eos), master cut its connection with client, free the worker-master connection, and wait for new client to connect?

IMHO, this could make the real number of available worker always consistent with expectation ?

thanks,

Best regards,

Li

alumae commented 6 years ago

This would require a significant rewrite which I'm not going to do.