streaming binary chunks from socket.io.js to flask-socketio

[BRIDGE-AI]

@miguelgrinberg

I'm integrating google speech api on my flask-socketio server. This implementation is based on flask-socketio and gunicorn with gevent-websocket. I reduced transport method only to websocket on client side socket.io.js.

var socket = io(host, {
    transports: ['websocket'],
    reconnection: true,
    autoConnect: true,
});

And it emits microphone audio stream as binary data every 4KB.

var scriptNode = audioCtx.createScriptProcessor(4096, 1, 1);

const MAX_INT = Math.pow(2, 16 - 1) - 1;

scriptNode.addEventListener('audioprocess', function(e) {
    var floatSamples = e.inputBuffer.getChannelData(0);

    // The samples are floats in range [-1, 1]. Convert to 16-bit signed integer.
    var int16array = Int16Array.from(floatSamples.map(function(n) {
        return n * MAX_INT; 
    }));
    socket.binary(true).emit('asr-recognize', {stream:int16array});
});

I have a sample project for it in java version below.

https://codelabs.developers.google.com/codelabs/speaking-with-a-webpage/#0

And I found similar server-side code for python version below.

https://gist.github.com/cobookman/6459f0423d56527ad136999e57d181ea#file-server-py-L154

Here is my question. Now I think I need a method like websocket.recv() on flask-socketio. (on line 154) Is it supported or possible now? Does flask-socketio or python-socketio have it?

Or how can I conquer this problem? Do you have any idea?

I need your help again, like always. Thank you, miguel.

[miguelgrinberg]

Socket.IO works with event handlers, it does not have a blocking recv function. You are sending your data in the asr-recognize event, so you need to add a handler for that event in your server. This handler will be invoked every time data arrives.