civetweb / civetweb

Embedded C/C++ web server
Other
2.9k stars 962 forks source link

mg_websocket_write() returns different size than what I try to send #1218

Open mfide opened 11 months ago

mfide commented 11 months ago

Hello,

We have an embedded websocket server application uses Linux Kernel v5.4 and CivetWeb v1.16 with 4GB RAM.

Since this is a websocket protocol package based, not like pure TCP stream based, the server sends some frames to a client using mg_websocket_write() function, which each frame is about 1046 bytes in our case and we expect to receive whole frame at a time on the client side to process.

Whenever the server wants to send a frame to a client, our code snippet for that uses mg_websocket_write() function and we got an error reported something like that: length is 1046 but sent is 390 time to time. (We don't have SSL for this particular connection)

...
    sent = mg_websocket_write(ch->conn, opcode, (const char *)data, length);
    if ((sent < 0) || ((size_t)sent != length)) {
        CONSOLE_ERROR("websocket mg_websocket_write() for %s to %s:%d length %zu returned %d\n",
                      ch->ctx.uri, rq->remote_addr, rq->remote_port, length, sent);
        ret = false;
        goto out_unlock;
    }
...

Some logs I got from above code:

websocket mg_websocket_write() for /pro to 192.168.44.12:59711 length 1046 returned 390 websocket mg_websocket_write() for /pro to 192.168.44.12:59711 length 1046 returned -1 websocket mg_websocket_write() for /pro to 192.168.44.12:59711 length 1046 returned -1 websocket mg_websocket_write() for /pro to 192.168.44.12:59711 length 1046 returned -1 websocket mg_websocket_write() for /pro to 192.168.44.12:59711 length 1046 returned -1

Then our code is designed to terminate the active client connection in case of such length mismatches.

If I'm not mistaken, I remember that this was not the exact behaviour of previous CivetWeb versions with Linux kernel v4.4. If there was no enough buffer to send 1046 byte frame at a time, it was blocking. As I see this is not the case anymore?

This situation happens especially when client is not fast enough to process frames sent by the server.

Is this behaviour expected? What is the suggested workaround? Thanks.

mfide commented 11 months ago

For a workaround to this issue, I need to know available buffer size in the CivetWeb before calling mg_websocket_write() function. If available buffer size in the civetweb layer is less then my frame size that I wanna send, I can postpone calling mg_websocket_write() function to another time.

Do we have such a function in the library?