MQTT cannot receive large messages with TLS enabled (GIT8266O-676)

[janav1]

Environment

• Development Kit: ESP-WROOM02
• IDF version: v3.4
• Development Env: Eclipse IDE + Make
• Operating System: Windows 10 x64
• Power Supply: external 5V

Problem Description

We have successfully implemented MQTT OTA without secure connection. Everything worked like a charm until we enabled the secure connection (TLS) as described here: #https://github.com/espressif/ESP8266_RTOS_SDK/issues/1090 With TLS enabled and working, there is no possibility to receive larger MQTT messages. The connection breaks and must be re-established (see log).

Expected Behavior

Receive multiple MQTT_EVENT_DATA events with different message offset and length like without TLS enabled.

Actual Behavior

MQTT connection disconnects unexpectedly.

Steps to repropduce

use MQTT client with TLS enabled, subscribe to topic and receive a large MQTT message.

Code to reproduce this issue

part of static esp_err_t mqtt_event_handler_cb(esp_mqtt_event_handle_t event)

...
case MQTT_EVENT_ERROR:
        printf("MQTT_EVENT_ERROR\r\n");
        if (event->error_handle->error_type == MQTT_ERROR_TYPE_ESP_TLS) {
            printf("Last error from esp-tls: 0x%x", event->error_handle->esp_tls_last_esp_err);
            printf("Last tls stack error number: 0x%x", event->error_handle->esp_tls_stack_err);
        }
break;
...

Debug Logs

ESP side after receive OTA request:

Begin OTA, length: 740560 B
// Prepare partition
Writing to partition ota_1 at offset 0x110000
OTA begin succeeded
// Partition ready, waiting for firmware data
MQTT_EVENT_ERROR
Last error code reported from esp-tls: 0x0
Last tls stack error number: 0x0
MQTT_EVENT_DISCONNECTED
MQTT: trying to reconnect
MQTT_EVENT_BEFORE_CONNECT
MQTT_EVENT_CONNECTED
MQTT: sending subscribe
MQTT_EVENT_SUBSCRIBED

[janav1]

This is caused by a insufficient RAM because default TLS buffers are 16 kB. Solution might be to set TLS MAX_FRAGMENT_LENGTH, but TLS will not negotiate this size reduction with the server. Only sets the size of the receive / send internal buffers. According to Mbed TLS, the library supports this extension. As you can see from the Wireshark log, the server has no idea about resizing the fragment and still uses default value (16 kB). The server must use default value and the client must negotiate different max fragment length. Any chance to solve this issue?

Wireshark log: TCP payload (243 bytes) Transport Layer Security TLSv1.2 Record Layer: Handshake Protocol: Client Hello Content Type: Handshake (22) Version: TLS 1.0 (0x0301) Length: 238 Handshake Protocol: Client Hello Handshake Type: Client Hello (1) Length: 234 Version: TLS 1.2 (0x0303) Random: 00001402bd92d513e58a7199c2276ca78bbe89692fa6fb7e9c458830c8fc4d73 Session ID Length: 0 Cipher Suites Length: 98 Cipher Suites (49 suites) Compression Methods Length: 1 Compression Methods (1 method) Extensions Length: 95 Extension: server_name (len=19) Extension: signature_algorithms (len=22) Extension: supported_groups (len=24) Extension: ec_point_formats (len=2) Extension: encrypt_then_mac (len=0) Extension: extended_master_secret (len=0) Extension: session_ticket (len=0)