When connecting to flespi.io MQTT overSecure WebSocket, I get "TRANSPORT_WS: Sec-WebSocket-Accept not found" and "MQTT_CLIENT: Error transport connect" error messages (IDFGH-1146)

[ipapp]

Environment

• Development Kit: [ESP32-Wrover-Kit]
• Kit version (for WroverKit/PicoKit/DevKitC): [not sure]
• Module or chip used: [ESP32-WROVER-B]
• IDF version: v4.0-dev-512-g6fd535c98, but happened with earlier also
• Build System: [Make]
• Compiler version : 5.2
• Operating System: Linux
• Power Supply: USB

Problem Description

I am trying to connect to flespi.io over Secure WebSocket. The WS upgrade process fails, as the first upgrade response is not received in single esp_transport_read function call. It is not up to the wrong case of Sec-WebSocket-Accept.

Expected Behavior

I would expect to connect to wss://mqtt.flespi.io. It uses port 443. Other clients I made using e.g. node.js works fine when supplied the same credentials and URL.

Actual Behavior

I get the following errors:

TRANSPORT_WS: Sec-WebSocket-Accept not found
MQTT_CLIENT: Error transport connect

And the board cannot connect to wss://flespi.io:xxxx.

Steps to reproduce

1. Create an WSS MQTT client to connect to wss://flespi.io:443
2. Open a free account on flespi.
3. Try to connect to flespi MQTT server.

Analysis

The issue seems to be in the ws_connect function in transport_ws.c, line 116.

The esp_transport_read function at first call returns 34 bytes, which is not the whole response. Empirically I figured out that the response is 159 bytes. If I add a code like this, it works.

    memset(ws->buffer, 0, DEFAULT_WS_BUFFER);
    int len1 = 0;
    len = 0;
    while (len < 159 && len1 >= 0) {
        len1 = esp_transport_read(ws->parent, ws->buffer + len, DEFAULT_WS_BUFFER - len, timeout_ms);
        len += len1;
        ESP_LOGI(TAG, "Chunk %d", len1);
    }
    if (len == 0 && len1 < 0) {
        ESP_LOGE(TAG, "Error read response for Upgrade header %s", ws->buffer);
        return -1;
    } else {
        ESP_LOGI(TAG, "Response for Upgrade header [%d]:\n\r%s", len, ws->buffer);
    }

It results in 3 calls, returning 34, 123 and 2 bytes (159 in total). This is not a solution, as 159 byte response is likely just a special case for flespi.io! My feeling is that:

• either the esp_transport_read exists prematurely,
• or if it behaves as expected, then ws_connect has to handle the fragmented response.

Code to reproduce this issue

    esp_mqtt_client_config_t mqtt_cfg;
    memset(&mqtt_cfg, 0, sizeof(mqtt_cfg));
    mqtt_cfg.event_handle = mqtt_event_handler;
    mqtt_cfg.uri = "wss://mqtt.flespi.io";
    mqtt_cfg.client_id = m_Username.c_str();
    mqtt_cfg.username = FLESPI_TOKEN;
    mqtt_cfg.password = "";
    mqtt_cfg.user_context = this;
    mqtt_cfg.keepalive = 30;
    esp_mqtt_client_init(&mqtt_cfg);

Debug Logs

My extended log on error, when esp_transport_read is called only once in ws_connect, line 116.

E (17988) TRANSPORT_WS: Sec-WebSocket-Accept not found in response [34]:
HTTP/1.1 101 Switching Protocols

E (17988) MQTT_CLIENT: Error transport connect

[david-cermak]

Hi @ipapp

Thanks for reporting this issue! Indeed there's a bug in ws_transport, so that ws header is not received properly if fragmented. Correct fix should be very close to your proposed workaround (just checking for http header termination instead of fixed size):

    int header_len = 0;
    do {
        if ((len = esp_transport_read(ws->parent, ws->buffer + header_len, DEFAULT_WS_BUFFER - header_len, timeout_ms)) <= 0) {
            ESP_LOGE(TAG, "Error read response for Upgrade header %s", ws->buffer);
            return -1;
        }
        header_len += len;
        ws->buffer[header_len] = '\0';
        ESP_LOGD(TAG, "Chunk %d", header_len);
    } while (NULL == strstr(ws->buffer, "\r\n\r\n"));

Thanks again, will be fixed.

[ipapp]

Great, thanks! I'll try it in the coming days. I have one question: in case the response is wrong, will the esp_transport_read take care of timeout and/or error handling?

[david-cermak]

@ipapp Yes, esp_transport_read return <=0 in case of timeout or error, so returns to the caller (mqtt connect, which prints the errno). Also, this is just a quick fix, the actual one will be merged in coming days.

[ghaneipour]

hello i use B4A it not support WSS for MQTT can help me?

[ghaneipour]

ghaneipour@gmail.com