mqtt/tls configuration error

[pawelm87]

Hi, I have a problem establishing an MQTT(PubSubClient) connection using TLS over the W5500 ethernet with ESP32-S3. When I remove TLS, everything works, I have communication via MQTT.

First I connect one certificate (piece of the main code):

#include "Ethernet.h"
#include <w5500.h>
#include <ESP_SSLClient.h>
#include <PubSubClient.h>
...
Ethernet ethernet;
EthernetClient client;
ESP_SSLClient sslClient;
PubSubClient mqttClient(sslClient;)
...
const char clientCa2[] PROGMEM = R"(.........) // from provider side
...
void setup()
{
// W5500 ethernet initialisation mac configuration etc. default chip/library settings
...
  sslClient.setInsecure();
  sslClient.setCertificate(clientCa2); // from test broker site test.mosquitto.org
  sslClient.setBufferSizes(1024, 512);
  sslClient.setDebugLevel(4);
  sslClient.setClient(&client, true);

  mqttClient.setServer("test.mosquitto.org", 8883); // test broker
);

void loop()
{
    if (!mqttClient.connected()) {
      while (!mqttClient.connected()) {
        if (mqttClient.connect("123456")) {
          mqttClient.subscribe("status", 0);
          delay(1000);
          Serial.println("publich message");
          mqttClient.publish("status", "message ssl");
          delay(1000);
        } else {
          Serial.printf("connect failed status: %d", mqttClient.state());
        }
        delay(5000);
      }
    } else {
      Serial.println("connection failed";)
    }
);

logs:

INFO.mConnectBasicClient: Basic client connected! INFO.mConnectSSL: Start connection. INFO.mConnectSSL: Wait for SSL handshake. INFO.mUpdateEngine: State RECVREC INFO.mUpdateEngine: State RECVREC INFO.mUpdateEngine: State Connection close WARN.mRunUntil: Terminating because the ssl engine closed. ERROR.mConnectSSL: Failed to initlalize the SSL layer. ERROR.mConnectSSL: Incoming record or message has wrong type with regards to the current engine state.

If I switch to WIFI and use WiFiClientSecure and perform functions with the same names, everything works on the new encryption client.

Ultimately, I would like to connect to AWS and there it has 3 certificates (from the AWS website) I just add the certs. In addition, I have read that you have to set the correct clock:

#include <EthernetUdp.h>
#include <NTPClient.h>

EthernetUDP ethUdp;
NTPClient timeClient(ethUdp);

  timeClient.begin();
  timeClient.update();
  time_t t = timeClient.getEpochTime();
  Serial.println(t);
  sslClient.setX509Time(t);

and now I attach a new certificate:

  // sslClient.setInsecure();
  sslClient.setCACert(rootCA);
  sslClient.setCertificate(clientCa2);
  sslClient.setPrivateKey(clientKey);

Logs:

same as prev and new

WARN.mRunUntil: Terminating because the ssl engine closed. ERROR.mConnectSSL: Failed to initlalize the SSL layer. ERROR.mConnectSSL: Caller-provided parameter is incorrect.

I've tried many options, maybe I'm doing something wrong I'm not setting something or not in the right order.

Can you help

[mobizt]

You have to choose only one, not both.

sslClient.setInsecure();
sslClient.setCertificate(clientCa2);

It does not make sense to set SSL certificate bypassing then set to verify.

[pawelm87]

//sslClient.setInsecure();
sslClient.setCertificate(clientCa2);

the result is the same ...

ERROR.mConnectSSL: Failed to initlalize the SSL layer. ERROR.mConnectSSL: Incoming record or message has wrong type with regards to the current engine state.

BTW I used both on the basis of an analysis of WiFiSecureClient, which both functions do (set int variables)

[pawelm87]

If I don't call the function: sslClient.setInsecure(); // I know I shouldn't use that or sslClient.allowSelfSignedCerts(); I get in addition:

test.mosquitto.org (1 cert):

INFO.mConnectBasicClient: Basic client connected! INFO.mConnectSSL: Start connection. WARN.mConnectSSL: Connection will fail, no authentication method is setup. INFO.mConnectSSL: Wait for SSL handshake. INFO.mUpdateEngine: State RECVREC. INFO.mUpdateEngine: State RECVREC INFO.mUpdateEngine: State Connection close WARN.mRunUntil: Terminating because the ssl engine closed. ERROR.mConnectSSL: Failed to initlalize the SSL layer. ERROR.mConnectSSL: Incoming record or message has wrong type with regards to the current engine state.

aws ( 3 certy):

INFO.mConnectBasicClient: Basic client connected! INFO.mConnectSSL: Start connection. WARN.mConnectSSL: Connection will fail, no authentication method configured. INFO.mConnectSSL: Wait for SSL handshake. INFO.mUpdateEngine: RECVREC status. INFO.mUpdateEngine: RECVREC status INFO.mRunUntil: SSL state changed. INFO.mRunUntil: State RECVREC INFO.mRunUntil: Expected number of bytes: 5 INFO.mRunUntil: Expected number of bytes: 1344 INFO.mUpdateEngine: State Connection closed WARN.mRunUntil: Terminated because the ssl engine has been closed. ERROR.mConnectSSL: Failed to initialise the SSL layer. ERROR.mConnectSSL: The parameter provided by the caller is invalid.

I guess I should use: sslClient.allowSelfSignedCerts(); ??

[mobizt]

The problem is about the w5500 buffer size.

[pawelm87]

Can you explain? How much memory do I need? Should I increase the memory size for this socket?

maximum I could set up 1 socket 16KB -> TX and 16KB -> RX or if I need more sockets then I have to reconfigure the W5500

[mobizt]

You should try to set BearSSL rx buffer to 16k first (first parameter of setBufferSizes) because sever may not support fragmentation.

[pawelm87]

I have increased the buffers:

sslClient.setCertificate(clientCa2); sslClient.allowSelfSignedCerts(); sslClient.setBufferSizes(16384, 16384); sslClient.setDebugLevel(4); sslClient.setClient(&client, true);

I now have a different message/logs:

INFO.mConnectBasicClient: Basic client connected! INFO.mConnectSSL: Start connection. INFO.mConnectSSL: Wait for SSL handshake. INFO.mUpdateEngine: State RECVREC INFO.mUpdateEngine: State RECVREC INFO.mUpdateEngine: State Connection close WARN.mRunUntil: Terminating because the ssl engine closed. ERROR.mConnectSSL: Failed to initlalize the SSL layer. ERROR.mConnectSSL: Peer's public key does not have the proper type or is not allowed for the requested operation.

I think I also need to change the buffer in the W5500 or is there no need ?

[Pablo2048]

At least you have to setup NTP client (and get the time from server before trying to make SSL connection), because SSL x502 certificate need to know actual time. On ESP32 there is no BearSSL, but mbedTLS which, unfortunately, doesn't support MFLN. But you can try to use Tasmota ESP32 Core which has BearSSL integrated...

[pawelm87]

At least you have to setup NTP client (and get the time from server before trying to make SSL connection), because SSL x502 certificate need to know actual time. On ESP32 there is no BearSSL, but mbedTLS which, unfortunately, doesn't support MFLN. But you can try to use Tasmota ESP32 Core which has BearSSL integrated...

you are probably referring to this piece of code:

  timeClient.begin();
  timeClient.update();
  time_t t = timeClient.getEpochTime();
  Serial.println(t);
  sslClient.setX509Time(t);

I got: 1719933632 -> Tuesday, July 2, 2024 3:20:32 PM

[Pablo2048]

I don't call any sslClient.setX509Time() when connecting to AWS. I think that the SSL layer internals use system internal time management.

[pawelm87]

It makes sense to set the time I assumed that these APIs do that but I could be wrong. Please give me a hint what function from Arduino I can do this will save me time in testing.

[Pablo2048]

I'm using it like this (it's ripped off from big project so you have to do some cleaning by yourself)

#include "NTPTime.hpp"
#include "ESPF.hpp"
#include <sysvars.hpp>
#if defined(ESP8266)
# include <time.h>
# include <sys/time.h>
# include <sntp.h>
# include <coredecls.h> // settimeofday_cb()
#else
#  include "esp_sntp.h"
#endif
#include <trace.h>

#if defined(ESP8266)
static void _ntp_set()
#else
static void _ntp_set(struct timeval *tv)
#endif
{

    ESPF::event(APPEVENT_NTPINSYNC);
}

void NTPTime::start()
{

    _ntpServer = svGetV<String>(F("ntp"));
#ifdef EMERGENCY_NTP
    if (_ntpServer.length() == 0) {
        _ntpServer = F(EMERGENCY_NTP);
        TRACE(TRACE_WARNING, F("NTP: Emergency NTP used!"));
    }
#endif
    if (_ntpServer.length()) {
        _timeZone = svGetV<String>(F("tzone"));
        int escpos = _timeZone.indexOf('^');
        if (0 <= escpos)
            _timeZone.remove(0, escpos + 1);
#if defined(ESP8266)
        sntp_stop();
        settimeofday_cb(_ntp_set);
        configTime(_timeZone.c_str(), _ntpServer.c_str());
        sntp_init();
#else
        sntp_set_time_sync_notification_cb(_ntp_set);
        configTzTime(_timeZone.c_str(), _ntpServer.c_str());
#endif
        TRACE(TRACE_INFO, "NTP: Waiting for NTP...");
    } else {
        TRACE(TRACE_ERROR, "NTP: NTP server missing!");
    }
}

NTPTime ntptime;

[pawelm87]

Ok will test this lead. Also, does the time zone affect the encryption ( server vs client location) probably need to be taken into account ?

Back to the topic of buffer size, can this also have an impact ? MQTT is a light protocol for iot but when I add encryption I have problems. I am curious how much extra data is supplied when encryption is added.

[Pablo2048]

I'm mainly using it without timezone so it runs on UTC. As I have observed the main data increase came from initial SSL handshake. When the connection is established the data amount is nearly the same as in unencrypted socket.

[pawelm87]

mobizt what these errors are about:

ERROR.mConnectSSL: Incoming record or message has wrong type with regards to the current engine state.

ERROR.mConnectSSL: Caller-provided parameter is incorrect.

ERROR.mConnectSSL: Peer's public key does not have the proper type or is not allowed for the requested operation.

[mobizt]

@pawelm87

I test in both insecure and root CA certificate verification connection via googleapis.com using W5500 and ESP32. The test results are ok.

I test test.mosquitto.org in insecure mode and it is ok. But when apply the root CA provided from test.mosquitto.org, I get the same error as you posted above.

I also test with this example, it gives an error too which it used to work, and certificate was not yet expired.

I try both WiFiClient and EthernetClient and it gives the same result.

But for aws, I never tried it.

I have no idea at the moment. It may relate to the BearSSL crypto bugs of memcpy memory address overlapping but I'm not sure, but the SSL/TLS version and buffer size are not related.

I will test with ESP8266 which uses pre-compile BearSSL tomorrow and update.

[mobizt]

Ok, I have done testing for test.mosquitto.org on port 8883 with WiFiClient and EthernetClient (W5500) on ESP32.

The device time was set correctly or set via setX509Time.

The root CA cert was assigned using setCACert or setTrustAnchors with X509List.

Don't set allowSelfSignedCerts and don't use setCertificate which is for client cert.

The results for both insecure and Root CA connection are ok.

The functions of this library are same as ESP8266 WiFiClientSecure class because they use BearSSL library.

I never test for aws because I don't use it.

FYI, The certificate in the library example cannot be used as it is revoked from the server.

[mobizt]

@pawelm87

sslClient.setCACert(rootCA);
sslClient.setCertificate(clientCa2);
sslClient.setPrivateKey(clientKey);

It should be ok if a key/certificate pair (clientKey and clientCa2) was signed using rootCA cert and rootCA key.

[pawelm87]

mobizt I have only one certificate on test.mosquitto.org and I think it is not root CA, and I assume that I should use the function sslClient.setCertificate.

Additionally if I pass a cert and set the time:

EthernetUDP ethUdp;
  NTPClient timeClient(ethUdp);

  timeClient.begin();
  timeClient.update();
  time_t t = timeClient.getEpochTime();
  sslClient.setX509Time(t);
  sslClient.setCertificate(clientCa2); // from test.mosquitto.org

or if I set up all the certificates

sslClient.setCACert(rootCA); // aws case
sslClient.setCertificate(clientCa); // aws case diff than clientCa2
sslClient.setPrivateKey(clientKey); // aws case

or

sslClient.setCACert(clientCa2); // from test.mosquitto.org

and don't call the function sslClient.allowSelfSignedCerts() I get the error Connection *will* fail, no authentication method is setup.

Today it works for you on ESP32 and W5500 ?

[mobizt]

It does not matter, you can seะ server certificate to root CA using setCACert but not with setCertificate.

The certificate provided from https://[test.mosquitto.org](https://test.mosquitto.org/) is from free CA certificate i.e. Let's Encrypt.

The encrypted ports support TLS v1.3, v1.2 or v1.1 with x509 certificates and require client support to connect. For ports 8883 and 8884 you should use the certificate authority file (mosquitto.org.crt (PEM format), or mosquitto.org.der (DER format)) to verify the server connection.

Yes, it works for me as I post above.

The library is actually modified from ESP8266 WiFiClientSecure and OpeN Labs's SSLClient and that is why it should work.

[pawelm87]

Yesterday you wrote that you have a similar behaviour to me and today that it works, something has changed? Please, can you post the code you are testing with cert to test.mosquitto.org for ESP32 and W5500.

[mobizt]

Yesterday you wrote that you have a similar behaviour to me and today that it works, something has changed? Please, can you post the code you are testing with cert to test.mosquitto.org for ESP32 and W5500.

I take wrong certificate to test which is root CA (ISRG Root X1) which the subdomain test.mosquitto.org free certificate from Let's Encrypt is not in its chain.

[mobizt]

You have to use Let's Encrypt certificate with setCACert.

The setCACert is from ESP32 WiFiClientSecure style function which the function setTrustAnchors was called internally.

[pawelm87]

mobizt Please post the full code for the tested case for test.mosquitto.org (ESP32 and W5500) and link for certs (possibly char[] in the code). This would be very helpful.

[mobizt]

@pawelm87

I said wrong that the ca cert to test is from Lets Encrypt but it is from Mosquito itself.

Sorry for that I am a little bit busy.

Here is the code I test which I take from your post and from library example.

#include <Arduino.h>
#include <PubSubClient.h>
#include <Ethernet.h> //v2.0.2
#include <ESP_SSLClient.h>

// The mosquitto.org server certificate
// Expired June 7, 2030
const char caCert[] PROGMEM = "-----BEGIN CERTIFICATE-----\n"
                              "MIIEAzCCAuugAwIBAgIUBY1hlCGvdj4NhBXkZ/uLUZNILAwwDQYJKoZIhvcNAQEL\n"
                              "BQAwgZAxCzAJBgNVBAYTAkdCMRcwFQYDVQQIDA5Vbml0ZWQgS2luZ2RvbTEOMAwG\n"
                              "A1UEBwwFRGVyYnkxEjAQBgNVBAoMCU1vc3F1aXR0bzELMAkGA1UECwwCQ0ExFjAU\n"
                              "BgNVBAMMDW1vc3F1aXR0by5vcmcxHzAdBgkqhkiG9w0BCQEWEHJvZ2VyQGF0Y2hv\n"
                              "by5vcmcwHhcNMjAwNjA5MTEwNjM5WhcNMzAwNjA3MTEwNjM5WjCBkDELMAkGA1UE\n"
                              "BhMCR0IxFzAVBgNVBAgMDlVuaXRlZCBLaW5nZG9tMQ4wDAYDVQQHDAVEZXJieTES\n"
                              "MBAGA1UECgwJTW9zcXVpdHRvMQswCQYDVQQLDAJDQTEWMBQGA1UEAwwNbW9zcXVp\n"
                              "dHRvLm9yZzEfMB0GCSqGSIb3DQEJARYQcm9nZXJAYXRjaG9vLm9yZzCCASIwDQYJ\n"
                              "KoZIhvcNAQEBBQADggEPADCCAQoCggEBAME0HKmIzfTOwkKLT3THHe+ObdizamPg\n"
                              "UZmD64Tf3zJdNeYGYn4CEXbyP6fy3tWc8S2boW6dzrH8SdFf9uo320GJA9B7U1FW\n"
                              "Te3xda/Lm3JFfaHjkWw7jBwcauQZjpGINHapHRlpiCZsquAthOgxW9SgDgYlGzEA\n"
                              "s06pkEFiMw+qDfLo/sxFKB6vQlFekMeCymjLCbNwPJyqyhFmPWwio/PDMruBTzPH\n"
                              "3cioBnrJWKXc3OjXdLGFJOfj7pP0j/dr2LH72eSvv3PQQFl90CZPFhrCUcRHSSxo\n"
                              "E6yjGOdnz7f6PveLIB574kQORwt8ePn0yidrTC1ictikED3nHYhMUOUCAwEAAaNT\n"
                              "MFEwHQYDVR0OBBYEFPVV6xBUFPiGKDyo5V3+Hbh4N9YSMB8GA1UdIwQYMBaAFPVV\n"
                              "6xBUFPiGKDyo5V3+Hbh4N9YSMA8GA1UdEwEB/wQFMAMBAf8wDQYJKoZIhvcNAQEL\n"
                              "BQADggEBAGa9kS21N70ThM6/Hj9D7mbVxKLBjVWe2TPsGfbl3rEDfZ+OKRZ2j6AC\n"
                              "6r7jb4TZO3dzF2p6dgbrlU71Y/4K0TdzIjRj3cQ3KSm41JvUQ0hZ/c04iGDg/xWf\n"
                              "+pp58nfPAYwuerruPNWmlStWAXf0UTqRtg4hQDWBuUFDJTuWuuBvEXudz74eh/wK\n"
                              "sMwfu1HFvjy5Z0iMDU8PUDepjVolOCue9ashlS4EB5IECdSR2TItnAIiIwimx839\n"
                              "LdUdRudafMu5T5Xma182OC0/u/xRlEm+tvKGGmfFcN0piqVl8OrSPBgIlb+1IKJE\n"
                              "m/XriWr/Cq4h/JfB7NTsezVslgkBaoU=\n"
                              "-----END CERTIFICATE-----\n";

#define WIZNET_RESET_PIN 26 // Connect W5500 Reset pin to GPIO 26 of ESP32
#define WIZNET_CS_PIN 5     // Connect W5500 CS pin to GPIO 5 of ESP32
#define WIZNET_MISO_PIN 19  // Connect W5500 MISO pin to GPIO 19 of ESP32
#define WIZNET_MOSI_PIN 23  // Connect W5500 MOSI pin to GPIO 23 of ESP32
#define WIZNET_SCLK_PIN 18  // Connect W5500 SCLK pin to GPIO 18 of ESP32

uint8_t Eth_MAC[] = {0x02, 0xF0, 0x0D, 0xBE, 0xEF, 0x01};

ESP_SSLClient ssl_client;

EthernetClient basic_client;

PubSubClient mqttClient(ssl_client);

void ResetEthernet()
{
    Serial.println("Resetting WIZnet W5500 Ethernet Board...  ");
    pinMode(WIZNET_RESET_PIN, OUTPUT);
    digitalWrite(WIZNET_RESET_PIN, HIGH);
    delay(200);
    digitalWrite(WIZNET_RESET_PIN, LOW);
    delay(50);
    digitalWrite(WIZNET_RESET_PIN, HIGH);
    delay(200);
}

void networkConnection()
{

    Ethernet.init(WIZNET_CS_PIN);

    ResetEthernet();

    Serial.println("Starting Ethernet connection...");
    Ethernet.begin(Eth_MAC);

    unsigned long to = millis();

    while (Ethernet.linkStatus() == LinkOFF || millis() - to < 2000)
    {
        delay(100);
    }

    if (Ethernet.linkStatus() == LinkON)
    {
        Serial.print("Connected with IP ");
        Serial.println(Ethernet.localIP());
    }
    else
    {
        Serial.println("Can't connect");
    }
}

void setup()
{
    Serial.begin(115200);

    networkConnection();

    ssl_client.setCACert(caCert);

    ssl_client.setBufferSizes(1024 /* rx */, 512 /* tx */);

    ssl_client.setDebugLevel(4);

    ssl_client.setClient(&basic_client);

    // I hard code to set server verification timestamp for easy demonstation
    // and we use Ethernet which it may need EthernetUDP to access NTP server.
    ssl_client.setX509Time(1719989338);

    mqttClient.setServer("test.mosquitto.org", 8883);
}

void loop()
{
    if (!mqttClient.connected())
    {
        while (!mqttClient.connected())
        {
            if (mqttClient.connect("123456"))
            {
                mqttClient.subscribe("status", 0);
                delay(1000);
                Serial.println("publich message");
                mqttClient.publish("status", "message ssl");
            }
            else
            {
                Serial.printf("connect failed status: %d\n", mqttClient.state());
            }
        }
    }

    delay(5000);
}

[mobizt]

@pawelm87

The cert is already in the link from the text quoted from test.mosquito.org website I post above.

Or take here https://test.mosquitto.org/ssl/mosquitto.org.crt

[pawelm87]

Thanks for the code :) I wanted to synchronise that we are running on the same certs and library configuration.

for your code

  sslClient.setCACert(caCert); // yours const char caCert is the same as mine clientCa2

  sslClient.setBufferSizes(1024, 512);

  sslClient.setDebugLevel(4);

  sslClient.setClient(&client, true);

   sslClient.setX509Time(1719991100);

  mqttClient.setServer("test.mosquitto.org", 8883);

I get the following errors

INFO.mConnectBasicClient: Basic client connected! INFO.mConnectSSL: Start connection. > WARN.mConnectSSL: Connection will fail, no authentication method is setup. INFO.mConnectSSL: Wait for SSL handshake. INFO.mUpdateEngine: State RECVREC INFO.mUpdateEngine: State RECVREC INFO.mUpdateEngine: State Connection close WARN.mRunUntil: Terminating because the ssl engine closed. ERROR.mConnectSSL: Failed to initlalize the SSL layer. > ERROR.mConnectSSL: Incoming record or message has wrong type with regards to the current engine state.

the W5500 configuration is working fine because I can access port 1883 normally if I disable TLS

[mobizt]

Some users also faced this similar issue due to the W5500 buffers variation. I discussed this issue in Email library repo many years ago.

I use Arduino Ethernet library v2.0.2 ~instead of ESP32 core Ethernet library which is now ETH.h in ESP32 core v3.0.2~.

~I think the wrong Ethernet library may cause this issue?~

Edit: This is my faults it is different library between Ethernet.h and ETH.h.

[mobizt]

The circular buffer size was set in Ethernet.h based on the MAX_SOCK_NUM and ETHERNET_LARGE_BUFFERS.

https://github.com/arduino-libraries/Ethernet/blob/63eec8690b38639642b8adcabf9e356b3f97621b/src/utility/w5100.cpp#L86-L198

You can try config it.

The OpeN Labs's SSLClient recommend to use EthernetLarge library.

But I use default config of Ethernet.h library in my test in PlatformIO and it works fine.

[pawelm87]

I use the Ethernet.h (2.0.1) library from https://www.arduino.cc/en/Reference/Ethernet

I don't think there is that much difference between 2.0.1 and 2.0.2

[mobizt]

@pawelm87 The same library.

[mobizt]

@pawelm87

If you change to use WiFiClient, it should work too.

Other factors are based on the systems e.g. SPI speed, circuit, timing and power supply.

[mobizt]

This library can work with enc28j60 module too (requires some ethernet library).