Binary data limit

[owenmcateer]

Hello I am using WebSockets to send RGB data to an ESP8266 that the get displayed on a LED matrix. The data is being sent as a binary data Uint8Array. Everything works perfectly until I start to get above 400 RGB LED. The ESP will just restart and I can't seem to find the cause. Am I hitting the limitations of the ESP or WebSockets?

The matrix size I'm having problems is 24x21 RGB = 1512 byte array. This is the JavaScript that sends the WS data. It waits to hear back from the ESP before sending another frame of data.

ws = new WebSocket(`ws://192.168.1.x`);
ws.binaryType = 'arraybuffer';

// Create array buffer
const buffer = new ArrayBuffer(1512);
const bytearray = new Uint8Array(buffer, 0, 1512);

// Fill array with color data
// ...

ws.send(bytearray);

The full code can be found here: https://github.com/owenmcateer/canvas-cast/blob/master/arduino/ESP8266-Cast-Canvas.ino

Is this simply too much data to transmit? I'm I going about this the wrong way?, should I not be using binary WS or Uint8Array? Thanks

[Links2004]

Hi,

please enable debug in the IDE and post the output. you may are out of RAM or the browser does some wired fragmenting of the packed.

[owenmcateer]

Below is the full debug output. The error appears to be readCb receive TIMEOUT! 2001 and without the full data frame it crashes.

[readCb] receive TIMEOUT! 2001
[WS][0][handleWebsocket] missing data!
[WS][0][handleWebsocket] clientDisconnect code: 1002

The Arduino code I have collecting the frame data is as follows:

    case WStype_BIN:
      // Serial.printf("[%u] get binary length: %u\n", num, length);
      for (int i = 0; i < length; i += 3) {
        leds[i / 3].setRGB(payload[i], payload[(i + 1)], payload[(i + 2)]);
      }

Should I look at switching to WStype_FRAGMENT? What is involved on the client side to send fragmented data?

Full debug output

[WS][0][handleWebsocketWaitFor][readCb] size: 8 ok: 1
[WS][0][handleWebsocket] ------- read massage frame -------
[WS][0][handleWebsocket] fin: 1 rsv1: 0 rsv2: 0 rsv3 0  opCode: 2
[WS][0][handleWebsocket] mask: 1 payloadLen: 1512
[readCb] n: zu t: 1512
:rd 72, 536, 464
:rdi 72, 72
:c0 72, 536
[readCb] receive TIMEOUT! 2001
[WS][0][handleWebsocket] missing data!
[WS][0][handleWebsocket] clientDisconnect code: 1002
[WS][0][sendFrame] ------- send message frame -------
[WS][0][sendFrame] fin: 1 opCode: 8 mask: 0 length: 2 headerToPayload: 0
[WS][0][sendFrame] pack to one TCP package...
[write] n: zu t: 4
:wr 4 4 0
:wrc 4 4 0
[WS][0][sendFrame] sending Frame Done (4056us).
:ur 1
:close
WS:dis
:del
[WS-Server][0] client disconnected.
[0] Disconnected!

[owenmcateer]

I'm not sure if it is important but these are the Arduino setting I have.

Board: NodeMCU 1.0 (ESP-12E Module)
Flash size: 4M (1M SPIFFS)
IwIP Variant: v2 Prebuild (MSS=536)
CPU Frequency: 160MHz

[Links2004]

Hi, looks like the TCP packet is truncated to the MSS of 536. which means that you need to fragment your data.

[readCb] n: zu t: 1512
:rd 72, 536, 464
:rdi 72, 72
:c0 72, 536
[readCb] receive TIMEOUT! 2001

this shows the the lib wants to read 1512 bytes but the TCP stack delivers only 536 byte. for the lib this is a error case that is why the connection is than closed.

[owenmcateer]

Does this mean the problem is on the JavaScript side and it is not fragmenting the data? Or do I need to switch to using WStype_FRAGMENT? I can't seem to find anything online about sending fragmented binary data with websockets.

[Links2004]

fragmenting on the websocket data is normally abstracted on the browser side and you dont have direct access too this in you JavaScript code. most likely the browser does not care about the low MSS.

too get it working I recommend to implement your own fragmenting on top if the websocket protocol. this can be as simple as using the first two bytes in the payload as index. and then simply sending the data splittend to over the websocket.

something like this:

if(length < 2 || ((length-2)%3 != 0)) return; 
uint16_t offset = payload[0] | payload[1] << 8;
for (int i = 2; i < length; i += 3) {
    leds[offset + (i / 3)].setRGB(payload[i], payload[(i + 1)], payload[(i + 2)]);
}

has the nice benefit that you can update only one LED if you like ;)

[owenmcateer]

That sounds like a great idea, thanks. I'll try this and post the code here if I get it working.

[hilmanzhy]

ASK I can send data n emit parameter to a server socket IO, but a server want to send data to client n I dont understand program to received data from server... how to received data/message from server to client esp8266 socket IO client,master? please help me