Assertion error when connecting to Flask-Socketio server on ESP32

[deepaerial]

I'm trying to connect with Micropython flashed ESP32 to my Flask-Socketio server running on gunicorn, but when connuecting I'm getting this error:

import usocketio.client
usocketio.client.connect('http://192.168.0.108:5000')

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "usocketio/client.py", line 80, in connect
  File "usocketio/client.py", line 67, in _connect_http
  File "usocketio/client.py", line 61, in _connect_http
AssertionError: 

It looks like it failing in _connect_http() function because length variable is None:

def _connect_http(hostname, port, path):
    """Stage 1 do the HTTP connection to get our SID"""
    try:
        sock = socket.socket()
        addr = socket.getaddrinfo(hostname, port)
        sock.connect(addr[0][4])

        def send_header(header, *args):
            if __debug__:
                LOGGER.debug(str(header), *args)

            sock.write(header % args + '\r\n')

        send_header(b'GET %s HTTP/1.1', path)
        send_header(b'Host: %s:%s', hostname, port)
        send_header(b'')

        header = sock.readline()[:-2]
        assert header == b'HTTP/1.1 200 OK', header

        length = None

        while header:
            header = sock.readline()[:-2]
            if not header:
                break

            header, value = header.split(b': ')
            header = header.lower()
            if header == b'content-type':
                assert value == b'application/octet-stream'
            elif header == b'content-length':
                length = int(value)

        assert length # <--- failing on this line

        data = sock.read(length)
        return decode_payload(data)

    finally:
        sock.close()

Any ideas what could be wrong here?

[danni]

I'm guessing you didn't get a content-length or the value was zero. Log the header you get.

[deepaerial]

Here is header I get from Flask-SocketIO server, as you said there is no Content-length header present in handshake:

HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Type: application/octet-stream
Date: Sun, 02 Jun 2019 12:27:29 GMT
Server: gunicorn/19.9.0
Set-Cookie: io=8624d8ff2faa484d9d4be3a5ad30c7cc
Transfer-Encoding: chunked

+-----------------------------------------+
| NOTE: binary data not shown in terminal |
+-----------------------------------------

I'm also referenced this issue here but according to author of Flask-SocketIO I'm using regular websockets when connecting to server. At this I kind of confused as I thought SocketIO uses http to perform initial handshake and switch to regular TCP.

[danni]

I think the Flask-SocketIO author is confused about what's happening here, but does this code work for you: https://github.com/danni/uwebsockets/blob/esp8266/examples/socketio_client.py ?

To be honest, I tested this against Flask-SocketIO at the time, but I don't remember the details. Also it was a while ago, and the spec is super vague, so it could have broken. You might have to go bisect the commit history to see if something broke it.

[danni]

Okay so I suspect the problem is that I never tested this against gunicorn. I was very lazy with my implementation of the HTTP spec. From the other thread:

The Content-Length header is not required in an HTTP response. Some web servers may add it on their own, and that is why at the time you tested this you may have seen it in responses. When length is not included, the client is supposed to read until the connection is closed.

So that will need implementing.

[deepaerial]

Okay, I understand. I've also tried this on uwsgi server but the outcome is the same. Is there any simple way to reimplement this?

[deepaerial]

I've tried to substitute data = sock.read(length) with data = sock.readline() and it looks like it works wit uwsgi. But on gunicorn server I recieve this error:

Traceback (most recent call last):
  File "main.py", line 26, in <module>
  File "main.py", line 8, in hello
  File "usocketio/client.py", line 81, in connect
  File "usocketio/protocol.py", line 43, in decode_payload
IndexError: memoryview index out of range

Also when server goes offline client can't reconnect to it when server goes back online:

Traceback (most recent call last):
  File "main.py", line 12, in periodic_task
  File "usocketio/transport.py", line 42, in emit
  File "usocketio/transport.py", line 125, in _send_message
  File "usocketio/transport.py", line 121, in _send_packet
  File "uwebsockets/protocol.py", line 229, in send
  File "uwebsockets/protocol.py", line 147, in write_frame
OSError: [Errno 104] ECONNRESET

[danni]

readline() isn't correct on binary data, so you probably don't have the whole packet, which is why it's dying. Try just plain read() which should be safe here. It might be enough to remove the assert and let read do its thing.

The second one is probably also something that needs handling better. I never had a case where the server was resetting my connection. You'll need to catch the exception and reconnect.

I'd love a pull request for both of these issues :)

[roshie]

I’m dealing with that problem, too. My gunicorn server is sending responses without the "Content-Length" Header. Got a workaround, but not sure how well it works.

In usocketio/client.py, line 61, replacing "assert length" with

data = b''        
if not length:
       # Ignore the first line of data
       sock.readline()
       while True:
             content = sock.readline()
             if not content or '}' in content:
                  data += content
                  break
             data += content
else:
       data = sock.read(length)

This worked for me. I'm an amateur in programming. Please forgive me if this didn't make any sense.

[mhanuel26]

Hi @roshie ,

I was trying to make this work too but I am getting a socket.io protocol mismatch

 * Restarting with stat
 * Debugger is active!
 * Debugger PIN: 129-419-466
(859951) wsgi starting up on http://0.0.0.0:5000
(859951) accepted ('192.168.1.23', 56988)
The client is using an unsupported version of the Socket.IO or Engine.IO protocols (further occurrences of this error will be logged with level INFO)
192.168.1.23 - - [29/Jun/2022 17:32:30] "GET /socket.io/?EIO=3 HTTP/1.1" 400 195 0.000146

I then changet the EIO=3 to 4 in line 76 of client.py and now I am getting response back from Flask-SocketIO but I believe the parser is broken, it looks the protocol is not very documented.

Now the thing is that I am getting a content type assertion in micropython code, the line 56 of client.py the following code is asserting

if header == b'content-type':
                assert value == b'application/octet-stream'
            elif header == b'content-length':
                length = int(value)

After inspecting the OK response from Flask, I can see that content type is plain text as you can see here

But I believe the function _decodepayload will not parse correctly this.

I was looking to install an older version of Flask-SOcketIO but it is just difficult to match the versions, I am getting any kind of weird errors.

How did you handle to make this work?

@danni Have you look at this recently, any comments maybe?

Thanks, Manuel