Azure Blob Storage data fetch fails due to HTTPS Client MBed TLS (IDFGH-11100) (CA-313)

Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

General issue report

Hi everybody,

I try to fetch a binary file from an Azure Blob Storage, using esp_http_client, IDF v5.0.1. The fetch works in Postman or in a browser, which means the url path and the SAS token are correct.

I have uploaded two .txt files (3kB & 12KB) in the Azure blob. The strange thing is that the small file (test.txt) is somehow loaded, even there is an error from the HTTP_CLIENT. The beginning of the text file appears in the Error log:

I (12172) OTA: Opened successfully the HTTP connection I (12252) HTTP_CLIENT: Body received in fetch header state, 0x3ffbad7a, 486 E (12252) HTTP_CLIENT: This is a test file. This example demonstrates common operations the OTA tool:

reading, writing and erasing OTA partitions,
switching boot partitions, and
switching to factory partition.

Users taking a look at this example should focus on the contents of the Python script otatool_example.py or shell script otatool_example.sh. The scripts contain programmatic invocation of the tool's functions via the Python API and command-lï¿½?! I (12302) OTA: Fetching the Image file of length 2840 bytes I (12302) OTA: Incremented length 1024 I (12312) OTA: Incremented length 2048 I (12312) OTA: Incremented length 2840 I (12322) OTA: Connection closed I (12322) OTA: Total received data length: 2840 I (12332) HTTP_CLIENT: HTTP_EVENT_DISCONNECTED

The second file: test_long.txt doesn't load at all, and throws the following error:

I (14202) OTA: Opened successfully the HTTP connection E (14282) esp-tls-mbedtls: read error :-0x7100: E (14282) transport_base: esp_tls_conn_read error, errno=Connection already in progress E (14292) OTA: HTTP client fetch headers failed I (14292) HTTP_CLIENT: HTTP_EVENT_DISCONNECTED

The same error appears in the case of any large size file. I have inlined the http client code below, which contains valid URLs for the test files, and SAS tokens with the expiration at the end of October, so plenty of time left for testing.

//////////////////////////////////////////////////////////////////////////////////// esp_http_client_config_t config = { .url = "https://dumstorage.blob.core.windows.net/test/test.txt?sp=r&st=2023-09-13T16:10:57Z&se=2023-10-31T01:10:57Z&spr=https&sv=2022-11-02&sr=b&sig=5WjKp2Mp7uJWVJNLr2sdYt17YWuhYpVoJSwhmmBBgAo%3D", //.url = "https://dumstorage.blob.core.windows.net/test/test_long.txt?sp=r&st=2023-09-13T14:51:07Z&se=2023-10-30T23:51:07Z&spr=https&sv=2022-11-02&sr=b&sig=lK88A5l41ysl%2BxEgwhvOsYZaia3CdJzFchDHnLlGaNE%3D", .transport_type = HTTP_TRANSPORT_OVER_SSL, .event_handler = http_event_handler, .cert_pem = rootCaPem, .cert_len = rootCaPemLen, .timeout_ms = 5000, //.auth_type = HTTP_AUTH_TYPE_BASIC, .keep_alive_enable = true, .skip_cert_common_name_check = true, //.use_global_ca_store = true };

    esp_http_client_handle_t client = esp_http_client_init(&config);
    if (client == NULL) {
       ESP_LOGE(TAG, "Failed to initialise HTTP connection");
       return;
    }

    //esp_http_client_set_header(client, "Authorization", "");
    //esp_http_client_set_header(client, "ContentType", "application/octet-stream");
    //esp_http_client_set_header(client, "Accept", "*/*");
    esp_http_client_set_method(client, HTTP_METHOD_GET);

    err = esp_http_client_open(client, 0);
    if (err != ESP_OK) {
      ESP_LOGE(TAG, "Failed to open HTTP connection: %s", esp_err_to_name(err));
      esp_http_client_cleanup(client);
      return;
    }else{
        ESP_LOGI(TAG, "Opened successfully the HTTP connection");
    }

    content_length = esp_http_client_fetch_headers(client);

    if (content_length <0){
        ESP_LOGE(TAG, "HTTP client fetch headers failed");
        esp_http_client_cleanup(client);
        return;
    }else{
        ESP_LOGI(TAG, "Fetching the Image file of length %d bytes", content_length);
    }

    int binary_file_length = 0;

    while (1) {
        int data_read = esp_http_client_read_response(client, ota_write_data, BUFFSIZE);

        if (data_read < 0) {
            ESP_LOGE(TAG, "Error: SSL data read error");
            esp_http_client_cleanup(client);
            return;
        } else if (data_read > 0) {
              //ESP_LOGI(TAG, "Read data: %s", ota_write_data);
            binary_file_length += data_read;
            ESP_LOGI(TAG, "Incremented length %d", binary_file_length);
        } else if (data_read == 0) {
            if (errno == ECONNRESET || errno == ENOTCONN) {
                ESP_LOGE(TAG, "Connection closed, errno = %d", errno);
                break;
            }
            if (esp_http_client_is_complete_data_received(client) == true) {
                ESP_LOGI(TAG, "Connection closed");
                break;
            }
        }
    }
    ESP_LOGI(TAG, "Total received data length: %d", binary_file_length);
    if (esp_http_client_is_complete_data_received(client) != true) {
        ESP_LOGE(TAG, "Error in receiving complete file");
        http_cleanup(client);
        return;
    }

    esp_http_client_cleanup(client);

////////////////////////////////////////////////////////////////////////////////////

Given that the code above is quite standard, I believe there must be some client config settings which are required in particular for the Azure blob storage data fetch. Any idea why it doesn't work?

Any help will be highly appreciated!

espressif / esp-azure

Azure Blob Storage data fetch fails due to HTTPS Client MBed TLS (IDFGH-11100) (CA-313) #136

Answers checklist.

General issue report