espressif / esp-protocols

Collection of ESP-IDF components related to networking protocols
198 stars 135 forks source link

Websocket: fragmented transfers are not serialized (IDFGH-14001) #687

Open bryghtlabs-richard opened 4 weeks ago

bryghtlabs-richard commented 4 weeks ago

Answers checklist.

General issue report

When multiple threads write to the websocket, and at least one uses a fragmented transfer, the transfers are not serialized, leading to mixed fragments, corruption, and sometimes connection drops.

If multiple threads using the websocket is not expected, please consider the ability to serialize messages as a feature-request, as sometimes my WEBSOCKET_EVENT_DATA needs to respond by writing to the websocket, and I do not believe this can be safely serialized externally. If I create an external lock to serialize websocket writers, and I need my WEBSOCKET_EVENT_DATA handler to respond to some events like Engine.IO ping, because WEBSOCKET_EVENT_DATA holds the client lock, it sometimes deadlocks when another thread tries to send.

Console log of two writer threads, each calling esp_websocket_client_send_text_partial(); esp_websocket_client_send_cont_msg(); esp_websocket_client_send_fin() corrupting the packet stream:

I (9763) websocket: WEBSOCKET_EVENT_CONNECTED
I (10183) websocket_client: Sending 32 bytes took 0.000 seconds #Thread1, esp_websocket_client_send_text_partial()
I (10183) websocket_client: Sending 32 bytes took 0.000 seconds #Thread2, esp_websocket_client_send_text_partial()
I (10243) websocket: WEBSOCKET_EVENT_DATA #Thread1, echo from esp_websocket_client_send_text_partial()
I (10243) websocket: Received opcode=8
W (10243) websocket: Received=�expected CONTINUATION, got <Opcode.TEXT: 1> #Thread2, echo from esp_websocket_client_send_text_partial()
W (10253) websocket: Total payload length=45, data_len=45, current payload offset=0
W (10263) transport_ws: esp_transport_ws_poll_connection_closed: unexpected data readable on socket=54
W (10263) websocket_client: Connection terminated while waiting for clean TCP close
I (10273) websocket: WEBSOCKET_EVENT_FINISH

Similar issue occurs when main thread calls esp_websocket_client_send_text_partial(); esp_websocket_client_send_cont_msg(); esp_websocket_client_send_fin() and WEBSOCKET_EVENT_DATA handler calls esp_websocket_client_send_text().

Example issue demonstrator project : https://github.com/bryghtlabs-richard/esp-protocols/tree/issue687demonstrator

Related to #625

bryghtlabs-richard commented 1 day ago

Please consider this a low-priority bug from our side - we can work with smaller fragments.