(Possible) Memory leak in connect/disconnect loop (CA-79)

[andresmanelli]

Hello,

I see that doing this kind of loop, the free heap decreases if an internet loss occurs in the ssl handshake step. After a few disconnections, this ends up in a device that can never again connect to AWS.

This is the snippet:

while (1) {
    ESP_LOGI(TAG, "Free memory: %d bytes", esp_get_free_heap_size());
    rc = aws_iot_shadow_connect(&mqttClient, &scp);
    if (SUCCESS != rc)
    {
        ESP_LOGE(TAG, "aws_iot_shadow_connect returned error %d", rc);
    }
    ESP_LOGI(TAG, "Free memory: %d bytes", esp_get_free_heap_size());
    vTaskDelay(pdMS_TO_TICKS(500));
    rc = aws_iot_shadow_disconnect(&mqttClient);
    if (SUCCESS != rc)
    {
        ESP_LOGE(TAG, "aws_iot_shadow_disconnect returned error %d", rc);
    }
    vTaskDelay(pdMS_TO_TICKS(500));
}

And here is the output, I commented the disconnection events.

(2609) shadow_manager: Free memory: 117904 bytes
(2979) wifi: Notification of a time synchronization event
(6869) shadow_manager: Free memory: 87896 bytes
(7889) shadow_manager: Free memory: 117232 bytes
(12189) shadow_manager: Free memory: 87900 bytes
(13209) shadow_manager: Free memory: 117236 bytes                        // DISCONNECTION
(21919) aws_iot: failed! mbedtls_ssl_handshake returned -0x6800
(21919) shadow_manager: aws_iot_shadow_connect returned error -4
(21919) shadow_manager: Free memory: 113768 bytes
(22439) aws_iot: failed! mbedtls_ssl_write returned -0x7100
(22939) shadow_manager: Free memory: 113776 bytes                       // RECONNECTION
(29799) shadow_manager: Free memory: 84460 bytes                        // Now Free heap is ~ 3400 less than ~ 117200
(30819) shadow_manager: Free memory: 113776 bytes
(34919) shadow_manager: Free memory: 84456 bytes
(35939) shadow_manager: Free memory: 113776 bytes                      // DISCONNECTION, but no handshake error
(54219) aws_iot: failed! mbedtls_net_connect returned -0x44
(54219) shadow_manager: aws_iot_shadow_connect returned error -24
(54219) shadow_manager: Free memory: 113776 bytes
(54739) aws_iot: failed! mbedtls_ssl_write returned -0x7100
(55239) shadow_manager: Free memory: 113776 bytes                       // RECONNECTION
(65439) shadow_manager: Free memory: 84460 bytes                        // Now Free heap is the same as before ~113700
(66459) shadow_manager: Free memory: 113776 bytes                       // DISCONNECTION
(75369) aws_iot: failed! mbedtls_ssl_handshake returned -0x6800
(75369) shadow_manager: aws_iot_shadow_connect returned error -4
(75369) shadow_manager: Free memory: 110424 bytes
(75889) aws_iot: failed! mbedtls_ssl_write returned -0x7100
(76389) shadow_manager: Free memory: 110424 bytes
(94719) aws_iot: failed! mbedtls_net_connect returned -0x44
(94719) shadow_manager: aws_iot_shadow_connect returned error -24
(94719) shadow_manager: Free memory: 110424 bytes
(95239) aws_iot: failed! mbedtls_ssl_write returned -0x7100
(95739) shadow_manager: Free memory: 110424 bytes                       // RECONNECTION
(102919) shadow_manager: Free memory: 81128 bytes                        // Now Free heap is ~ 3400 less than ~ 113700
(103939) shadow_manager: Free memory: 110424 bytes
(108239) shadow_manager: Free memory: 81128 bytes
(109259) shadow_manager: Free memory: 110424 bytes

Is there any way of cleaning things up ? I trying cleaning the underlying ssl context without luck.

Thank you !

[romocitto88]

Hi, I've a similar issue: if I try to connect and disconnect for n times from AWS Mqtt broker, sometimes the esp32 module reset when is executed "aws_iot_mqtt_disconnect(&hdl_mqtt_Client);" or loops into the function and never returns.

The error:

Disconnect Mqtt C:.platformio\packages\framework-espidf\components\freertos\queue.c:720 (xQueueGenericSend)- assert failed! abort() was called at PC 0x40098a7c on core 1

ELF file SHA256: f997dde20128e6df

Backtrace: 0x4008da79:0x3ffe2f60 0x4008ddfd:0x3ffe2f80 0x40098a7c:0x3ffe2fa0 0x4012f822:0x3ffe2fe0 0x40133210:0x3ffe3000 >0x4013331d:0x3ffe3030 0x40135dde:0x3ffe3050 0x40136801:0x3ffe3080 0x40130445:0x3ffe30a0 0x401304ec:0x3ffe30c0 >0x40098381:0x3ffe30f0

Rebooting...

Decoding stack results:

0x40133185: select_check_waiters at C:.platformio\packages\framework-espidf\components\lwip\lwip\src\api\sockets.c line 2672 0x4013331d: event_callback at C:.platformio\packages\framework-espidf\components\lwip\lwip\src\api\sockets.c line 2623 0x40135fa1: recv_tcp at C:.platformio\packages\framework-espidf\components\lwip\lwip\src\api\api_msg.c line 351 0x40124d2d: tcp_input at C:.platformio\packages\framework-espidf\components\lwip\lwip\src\core\tcp_in.c line 501 0x40129e16: ip4_input at C:.platformio\packages\framework-espidf\components\lwip\lwip\src\core\ipv4\ip4.c line 784 0x4012f6b2: ethernet_input at C:.platformio\packages\framework-espidf\components\lwip\lwip\src\netif\ethernet.c line 186 0x4013046d: tcpip_thread_handle_msg at C:.platformio\packages\framework-espidf\components\lwip\lwip\src\api\tcpip.c line 180 0x401304ec: tcpip_thread at C:.platformio\packages\framework-espidf\components\lwip\lwip\src\api\tcpip.c line 154 0x40098381: vPortTaskWrapper at C:.platformio\packages\framework-espidf\components\freertos\port.c line 143

The code:

void hdl_mqtt_deinit(void)
{
    printf("Delete Task\n");
    vTaskDelete(hdl_mqtt_taskHandler); //Delete the mqtt main task with yeld function
    printf("Disconnect Mqtt\n");
    IoT_Error_t err = aws_iot_mqtt_disconnect(&hdl_mqtt_Client); //Disconnect from AWS
    if(err!=SUCCESS)
        printf("Disc Unsuccess\n");
    else
        printf("Disc Success\n");
    err = aws_iot_mqtt_free(&hdl_mqtt_Client); //Free memory
    if(err!=SUCCESS)
        printf("free Unsuccess\n");
    else
        printf("free Success\n");
}

[carlosdomenje]

I have the same issue. In my case, device connect to wifi network to download data to aws. After n connections, never can connect to aws again. I put ESP_LOGW(TAG, "[AWS 9]- 1 Free memory: %d bytes", esp_get_minimum_free_heap_size()); in my code after rc = aws_iot_mqtt_connect(&client, &connectParams);

when i try to connect many times, heap memory is decreasing.

When I finish to download data to AWS, always i do this... aws_iot_mqtt_disconnect(&client); aws_iot_mqtt_free(&client);