search

espressif / arduino-esp32

Arduino core for the ESP32
GNU Lesser General Public License v2.1
13.74k stars 7.43k forks source link

Arduino Core 2.0.2 does not support TLS 1.3 --> WiFiClientSecure #6528

Closed martinius96 closed 2 years ago

martinius96 commented 2 years ago

Board

ESP32 Devkit V1 DOIT

Device Description

There is connected HC-SR04 ultrasonic sensor that have no impact on problem.

Hardware Configuration

D22 --> Trigger D23 --> Echo 3V3 --> Vcc GND --> GND

Version

v2.0.2

IDE Name

Arduino IDE 1.8.19

Operating System

Windows 10 Education x64

Flash frequency

80 MHz

PSRAM enabled

no

Upload speed

921600

Description

I have developed code in January for ESP32 that is using HTTPS connection via WiFiClientSecure library. It is doing measurements using HC-SR04 or JSN-SR04T ultrasonic sensor of water level height. Datas are sent to webserver, that is freehosting hosted at 000webhost --> https://hladinomer.000webhostapp.com/ I am not sure what version of Arduino Core I was using at development, maybe it was 2.0.2, but not sure...

Sketch is using FreeRTOS, there are other versions of sketch without FreeRTOS, but same problem. In FreeRTOS sketch there is used Queue. When datas are measured (once per 5 minutes), they are pushed to Queue, and HTTPS connection task will read them and will execute once to store them in webserver MySQL database.

In similar time as I developed Arduino Core, I have created identical sketch (from point of how it works) in ESP-IDF framework 4.4 on mbed_tls example and it is working without problem for months, so it doesn't have that issue... Right now I am using Arduino Core 2.0.2 - latest stable release. And code that was originally working in January is not working now. During SSL handshake there is ssl_client return -1, so certificate is not verified. Root CA certificate is same (DigiCert Global Root CA), also website certificate is valid for more than +120 days to go. So, nothing changed on that side for Secure client... Source code is correct.

Core 2.0.2 output with Core Debug level Verbose - Connection not working: WiFiClientSecure Arduino Core 2.0.2 I have tried WiFiClientSecure example and it is working good. When i compared both websites, there is difference at TLS version that is used on webservers. 000webhost is on subdomains using TLS 1.3 and 1.2 and howsmyssl.com (from WiFiClientSecure example) is using TLS 1.2.

So i think there might be problem with TLS 1.3 connection. Also in WiFiClientSecure of actual 2.0.2 release branch there is comment: "Running on TLS 1.2 using mbedTLS". Also I am not sure if 000webhost had TLS 1.3 before or it was added in recent days or months, I don't know how to find that out... Is it possible to force 1.2 TLS connection from side of ESP32?

How should I know if connection is made by TLS 1.3 or TLS 1.2? Debug giving me no output related to it

I tried to downgrade Arduino Core to 1.0.6. Then I tried same sketch and connection to 000webhost is working normally and datas are successfully sent to server where are stored in database. So I am a bit confused... I have tried to study documentation and also some Gitter threads, but nothing relevant I have found... So what should be wrong?

There is Wokwi simulator with lightweight sketch with one task that is trying to sent example datas to webservers without ultrasonic measurement - no luck on connection: https://wokwi.com/projects/327948646817464914

Core 1.0.6 output with Core Debug level Verbose - Connection WORKING! : WiFiClientSecure Arduino Core 1.0.6

Sketch

/*|-----------------------------------------------------------------------------------|*/
/*|Projekt: Hladinomer - HTTPS - FreeRTOS - HC-SR04 / JSN-SR04T / HY-SRF05            |*/
/*|ESP32 (DevKit, Generic)                                                            |*/
/*|Autor: Martin Chlebovec (martinius96)                                              |*/
/*|E-mail: martinius96@gmail.com                                                      |*/
/*|Info k projektu (schéma): https://martinius96.github.io/hladinomer-studna-scripty/ |*/
/*|Testovacie webove rozhranie: https://hladinomer.000webhostapp.com/                 |*/
/*|Licencia pouzitia: MIT                                                             |*/
/*|Revízia: 3. Január 2022                                                            |*/
/*|-----------------------------------------------------------------------------------|*/

#include <WiFi.h>
#include <WiFiClientSecure.h>
#include <NewPingESP8266.h>

const char * ssid = "MY_WIFI"; //MENO WiFi SIETE
const char * password = "MY_WIFI_PASSWORD"; //HESLO WiFi SIETE
const char* host = "hladinomer.000webhostapp.com"; //adresa webservera (doména) na ktorú sa odosielajú dáta
String url = "/data.php"; //URL adresa - cesta pod domenou k cieľovemu .php súboru, ktorý realizuje zápis

#define pinTrigger    22
#define pinEcho       23
#define maxVzdialenost 450
NewPingESP8266 sonar(pinTrigger, pinEcho, maxVzdialenost);

TaskHandle_t Task1; //ULTRASONIC MEASUREMENT
TaskHandle_t Task2; //WIFI HTTP SOCKET
QueueHandle_t  q = NULL;

WiFiClientSecure client;
static void Task1code( void * parameter);
static void Task2code( void * parameter);

//Root CA cert --> CERTIFIKÁT CERTIFIKAČNEJ AUTORITY, KTORÁ VYDALA CERTIFIKÁT VÁŠ WEBSERVER v .pem formáte
//DST ROOT CA X3 EXAMPLE (https://i.imgur.com/fvw4huT.png)
const static char* test_root_ca PROGMEM = \
    "-----BEGIN CERTIFICATE-----\n" \
    "MIIDrzCCApegAwIBAgIQCDvgVpBCRrGhdWrJWZHHSjANBgkqhkiG9w0BAQUFADBh\n" \
    "MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3\n" \
    "d3cuZGlnaWNlcnQuY29tMSAwHgYDVQQDExdEaWdpQ2VydCBHbG9iYWwgUm9vdCBD\n" \
    "QTAeFw0wNjExMTAwMDAwMDBaFw0zMTExMTAwMDAwMDBaMGExCzAJBgNVBAYTAlVT\n" \
    "MRUwEwYDVQQKEwxEaWdpQ2VydCBJbmMxGTAXBgNVBAsTEHd3dy5kaWdpY2VydC5j\n" \
    "b20xIDAeBgNVBAMTF0RpZ2lDZXJ0IEdsb2JhbCBSb290IENBMIIBIjANBgkqhkiG\n" \
    "9w0BAQEFAAOCAQ8AMIIBCgKCAQEA4jvhEXLeqKTTo1eqUKKPC3eQyaKl7hLOllsB\n" \
    "CSDMAZOnTjC3U/dDxGkAV53ijSLdhwZAAIEJzs4bg7/fzTtxRuLWZscFs3YnFo97\n" \
    "nh6Vfe63SKMI2tavegw5BmV/Sl0fvBf4q77uKNd0f3p4mVmFaG5cIzJLv07A6Fpt\n" \
    "43C/dxC//AH2hdmoRBBYMql1GNXRor5H4idq9Joz+EkIYIvUX7Q6hL+hqkpMfT7P\n" \
    "T19sdl6gSzeRntwi5m3OFBqOasv+zbMUZBfHWymeMr/y7vrTC0LUq7dBMtoM1O/4\n" \
    "gdW7jVg/tRvoSSiicNoxBN33shbyTApOB6jtSj1etX+jkMOvJwIDAQABo2MwYTAO\n" \
    "BgNVHQ8BAf8EBAMCAYYwDwYDVR0TAQH/BAUwAwEB/zAdBgNVHQ4EFgQUA95QNVbR\n" \
    "TLtm8KPiGxvDl7I90VUwHwYDVR0jBBgwFoAUA95QNVbRTLtm8KPiGxvDl7I90VUw\n" \
    "DQYJKoZIhvcNAQEFBQADggEBAMucN6pIExIK+t1EnE9SsPTfrgT1eXkIoyQY/Esr\n" \
    "hMAtudXH/vTBH1jLuG2cenTnmCmrEbXjcKChzUyImZOMkXDiqw8cvpOp/2PV5Adg\n" \
    "06O/nVsJ8dWO41P0jmP6P6fbtGbfYmbW0W5BjfIttep3Sp+dWOIrWcBAI+0tKIJF\n" \
    "PnlUkiaY4IBIqDfv8NZ5YBberOgOzW6sRBc4L0na4UU+Krk2U886UAb3LujEV0ls\n" \
    "YSEY1QSteDwsOoBrp+uvFRTp2InBuThs4pFsiv9kuXclVzDAGySj4dzp30d8tbQk\n" \
    "CAUw7C29C79Fv1C5qfPrmAESrciIxpg0X40KPMbp1ZWVbd4=\n" \
    "-----END CERTIFICATE-----\n";

void setup() {
  Serial.begin(115200);
  WiFi.begin(ssid, password); //pripoj sa na wifi siet s heslom
  while (WiFi.status() != WL_CONNECTED) {
    delay(1000);
    Serial.print(".");
  }
  Serial.println(F(""));
  Serial.println(F("Wifi connected with IP:"));
  Serial.println(WiFi.localIP());
  client.setCACert(test_root_ca);
  q = xQueueCreate(20, sizeof(int));
  if (q != NULL) {
    Serial.println(F("Queue FIFO buffer is created"));
    vTaskDelay(1000 / portTICK_PERIOD_MS); //wait for a second
    xTaskCreatePinnedToCore(
      Task1code,   /* Task function. */
      "Task1",     /* name of task. */
      10000,       /* Stack size of task */
      NULL,        /* parameter of the task */
      1,           /* priority of the task */
      &Task1,      /* Task handle to keep track of created task */
      1);          /* pin task to core 1 */
    Serial.println(F("Ultrasonic measurement task started"));
    xTaskCreatePinnedToCore(
      Task2code,   /* Task function. */
      "Task2",     /* name of task. */
      10000,       /* Stack size of task */
      NULL,        /* parameter of the task */
      1,           /* priority of the task */
      &Task2,      /* Task handle to keep track of created task */
      0);          /* pin task to core 0 */
    Serial.println(F("HTTP Socket task started"));
  } else {
    Serial.println(F("Queue creation failed"));
  }
}

void loop() {
  if (WiFi.status() != WL_CONNECTED) {
    WiFi.begin(ssid, password); //pripoj sa na wifi siet s heslom
  }
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  yield();

}

static void Task1code( void * parameter) {
  if (q == NULL) {
    Serial.println(F("Queue in Measurement task is not ready"));
    return;
  }
  while (1) {
    int distance = sonar.ping_cm();
    delay(50);
    Serial.print(F("Test measurement: "));
    Serial.print(distance);
    Serial.println(F(" cm"));
    if (distance > 0) {
      distance = 0;
      for (int i = 0; i < 10; i++) {
        distance += sonar.ping_cm();
        delay(50);
      }
      distance = distance / 10;
      Serial.print(F("Distance to water level is: "));
      Serial.print(distance);
      Serial.println(F(" cm."));
      xQueueSend(q, (void *)&distance, (TickType_t )0); //add the measurement value to Queue
      for (int countdown = 300; countdown >= 0; countdown--) {
        Serial.print(F("Next measurement in: "));
        Serial.print(countdown);
        Serial.println(F(" seconds"));
        vTaskDelay(1000 / portTICK_PERIOD_MS);
      }
    }
  }
}
static void Task2code( void * parameter) {
  int distance;
  if (q == NULL) {
    Serial.println(F("Queue in HTTP socket task is not ready"));
    return;
  }
  while (1) {
    xQueueReceive(q, &distance, portMAX_DELAY); //read measurement value from Queue and run code below, if no value, WAIT....
    String data = "hodnota=" + String(distance) + "&token=123456789";
    client.stop();
    if (client.connect(host, 443)) {
      Serial.println(F("Connected to server successfully"));
      client.println("POST " + url + " HTTP/1.0");
      client.println("Host: " + (String)host);
      client.println(F("User-Agent: ESP"));
      client.println(F("Connection: close"));
      client.println(F("Content-Type: application/x-www-form-urlencoded;"));
      client.print(F("Content-Length: "));
      client.println(data.length());
      client.println();
      client.println(data);
      Serial.println(F("Datas were sent to server successfully"));
      while (client.connected()) {
        String line = client.readStringUntil('\n');
        if (line == "\r") {
          break;
        }
      }
      String line = client.readStringUntil('\n');
    } else {
      Serial.println(F("Connection to webserver was NOT successful"));
    }
    client.stop();
  }
}

Debug Message

Copy from UART monitor is not working, I don't know why, I can just copy [ character...
I don't have ExceptionDecoder.

Other Steps to Reproduce

I tried Wokwi simulator, output is same. Link provided in Description. After downgrading Arduino Core to 1.0.6, connection is working without problem.

I have checked existing issues, online documentation and the Troubleshooting Guide

martinius96 commented 2 years ago

Edit 3 hours later: I tried to import .json of development Release branches to my Arduino IDE. Then in Board Manager I have used 2.0.3-RC1. And connection is working, that's shocking... WiFiClientSecure that is in Arduino Core for years... In one version working, in other not... That shouldn't happen in released stable versions... There is some SSL related error in UART output, but request was done okay and datas were written into MySQL database, success.

So tell me, how Arduino Core 2.0.2 can be based on ESP-IDF 4.4, if it is not using it...
Because as I can see, in ESP-IDF 4.4 it is working.
And at Arduino Core 2.0.3-RC1 that IS FINALLY BASED on v4.4 ESP-IDF it is working too.

ESP-IDF 4.4 sketch if you are interested (based on mbed_tls example): https://github.com/martinius96/hladinomer-studna-scripty/blob/master/examples/Hladinomer/HTTPS/https_mbedtls/main/https_mbedtls_example_main.c

In changelog of 2.0.3-RC1 there is: 

Support the updated MbedTLS in ESP-IDF v4.4 by @me-no-dev in https://github.com/espressif/arduino-esp32/pull/6243

Then why that wasn't uploaded in 2.0.2 STABLE version that is based on v4.4 ESP-IDF? You guys make me wrinkle and I lost few of my hair... Sad cat

Core 2.0.3-RC1 output with Core Debug level Verbose - Connection WORKING! : WiFiClientSecure Arduino Core 2.0.3-RC1

VojtechBartoska commented 2 years ago

Hello @martinius96, Arduino ESP32 Core v2.0.3-RC1 is based on ESP-IDF v4.4 and there have been done some fixes in meantime what can explain this behaviour.

We are now waiting for ESP-IDF v.4.4.1 which includes more bug fixes.

I'm sorry that it brings you some confusion.

In summary, under v2.0.3-RC1 everything is working correctly?

martinius96 commented 2 years ago

Thanks for reply, yes, under v2.0.3-RC1 WiFiClientSecure client working good, there is just that _handle_error() output -76 UNKNOWN ERROR CODE (004C), but request was successful and data written into MySQL database.

That v2.0.2 release was based on ESP-IDF v4.4-beta1, so that caused problem I think, because it wasn't from release ESP-IDF version. I haven't tried other things such as Bluetooth, UDP connections at WiFi or even buses, so I am not sure if there can be other things that are not working.

I have found out, sketch was originally developed and working under v2.0.1 release, that was based on stable ESP-IDF v4.4, not beta.

VojtechBartoska commented 2 years ago

@martinius96 Your specific issue could be solved by backports in ESP-IDF. Do you need more help on this? Seems sticking to 2.0.3 is a solution for this :)

martinius96 commented 2 years ago

Yes, 2.0.3 working great, we can close this issue, solved. Thanks for assistance @VojtechBartoska

CCarrillo07 commented 3 months ago

Hi all, I'm using esp32 by Espressif Systems version 2.0.3, I'm sending a GET request to www.howsmyssl.com.

I'm getting this response:

Response: {"given_cipher_suites":["TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384","TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384","TLS_DHE_RSA_WITH_AES_256_GCM_SHA384","TLS_ECDHE_ECDSA_WITH_AES_256_CCM","TLS_DHE_RSA_WITH_AES_256_CCM","TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384","TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384","TLS_DHE_RSA_WITH_AES_256_CBC_SHA256","TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA","TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA","TLS_DHE_RSA_WITH_AES_256_CBC_SHA","TLS_ECDHE_ECDSA_WITH_AES_256_CCM_8","TLS_DHE_RSA_WITH_AES_256_CCM_8","TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256","TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256","TLS_DHE_RSA_WITH_AES_128_GCM_SHA256","TLS_ECDHE_ECDSA_WITH_AES_128_CCM","TLS_DHE_RSA_WITH_AES_128_CCM","TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256","TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256","TLS_DHE_RSA_WITH_AES_128_CBC_SHA256","TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA","TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA","TLS_DHE_RSA_WITH_AES_128_CBC_SHA","TLS_ECDHE_ECDSA_WITH_AES_128_CCM_8","TLS_DHE_RSA_WITH_AES_128_CCM_8","TLS_RSA_WITH_AES_256_GCM_SHA384","TLS_RSA_WITH_AES_256_CCM","TLS_RSA_WITH_AES_256_CBC_SHA256","TLS_RSA_WITH_AES_256_CBC_SHA","TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384","TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384","TLS_ECDH_RSA_WITH_AES_256_CBC_SHA","TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384","TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384","TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA","TLS_RSA_WITH_AES_256_CCM_8","TLS_RSA_WITH_AES_128_GCM_SHA256","TLS_RSA_WITH_AES_128_CCM","TLS_RSA_WITH_AES_128_CBC_SHA256","TLS_RSA_WITH_AES_128_CBC_SHA","TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256","TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256","TLS_ECDH_RSA_WITH_AES_128_CBC_SHA","TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256","TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256","TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA","TLS_RSA_WITH_AES_128_CCM_8","TLS_EMPTY_RENEGOTIATION_INFO_SCSV"],"ephemeral_keys_supported":true,"session_ticket_supported":true,"tls_compression_supported":false,"unknown_cipher_suite_supported":false,"beast_vuln":false,"able_to_detect_n_minus_one_splitting":false,"insecure_cipher_suites":{},"tls_version":"TLS 1.2","rating":"Probably Okay"} Connection closed

I would like that esp32 supports TLS 1.3, Do you know how can I achieve this?