WiFi client enterprise breaks and reboot ESP32 trying to connect eduroam in version 3.0.0

[tutotio]

Board

ESP32 dev module

Device Description

Development kit

Hardware Configuration

Development board with nothing attached

Version

v3.0.0

IDE Name

Arduino IDE

Operating System

Windows 11

Flash frequency

80MHz

PSRAM enabled

no

Upload speed

921600

Description

WiFi client enterprise breaks and reboot ESP32 trying to connect eduroam in version 3.0.0 running examples:

https://github.com/espressif/arduino-esp32/blob/master/libraries/WiFi/examples/WiFiClientEnterprise/WiFiClientEnterprise.ino

https://github.com/espressif/arduino-esp32/blob/master/libraries/HTTPClient/examples/HTTPClientEnterprise/HTTPClientEnterprise.ino

It doesn't matter if you put a good or bad identity or password, it always breaks

It breaks if you are under wifi eduroam coberture. If eduroam wifi is not available or you change the ssid to a non existant ssid const char *ssid = "eduro";

you got an error, but it don't brake and reboot: [W][STA.cpp:135] _onStaArduinoEvent(): Reason: 201 - NO_AP_FOUND

Sketch

In the first example:

#include <WiFi.h>                      //Wifi library
#define EAP_IDENTITY "login"           //if connecting from another corporation, use identity@organization.domain in Eduroam
#define EAP_USERNAME "login"           //oftentimes just a repeat of the identity
#define EAP_PASSWORD "password"        //your Eduroam password
const char *ssid = "eduroam";          // Eduroam SSID
void setup() {
  WiFi.disconnect(true);  //disconnect form wifi to set new wifi connection
  WiFi.mode(WIFI_STA);    //init wifi mode
  WiFi.begin(ssid, WPA2_AUTH_PEAP, EAP_IDENTITY, EAP_USERNAME, EAP_PASSWORD);
}

In the second example:

#include <WiFi.h>
#include "esp_eap_client.h"
#define EAP_IDENTITY "identity"  //if connecting from another corporation, use identity@organization.domain in Eduroam
#define EAP_PASSWORD "password"  //your Eduroam password
const char *ssid = "eduroam";    // Eduroam SSID
void setup() {
  WiFi.disconnect(true);  //disconnect form wifi to set new wifi connection
  WiFi.mode(WIFI_STA);    //init wifi mode
  esp_eap_client_set_identity((uint8_t *)EAP_IDENTITY, strlen(EAP_IDENTITY));  //provide identity
  esp_eap_client_set_username((uint8_t *)EAP_IDENTITY, strlen(EAP_IDENTITY));  //provide username
  esp_eap_client_set_password((uint8_t *)EAP_PASSWORD, strlen(EAP_PASSWORD));  //provide password
  esp_wifi_sta_enterprise_enable();
  WiFi.begin(ssid);  //connect to wifi
}

Debug Message

Connecting to network: eduroam
[   754][W][STA.cpp:533] disconnect(): STA already disconnected.
[   778][V][NetworkEvents.cpp:119] checkForEvent(): Network Event: 9 - WIFI_READY
[   855][V][STA.cpp:184] _onStaEvent(): STA Started
[   859][V][NetworkEvents.cpp:119] checkForEvent(): Network Event: 11 - STA_START
[   867][D][STA.cpp:110] _onStaArduinoEvent(): Arduino STA Event: 11 - STA_START

assert failed: xQueueReceive queue.c:1475 (( pxQueue ))

Backtrace:
0x40082649: panic_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system\panic.c:466
0x4008bd85: esp_system_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/port\esp_system_chip.c:84
0x4009123e: __assert_func at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/newlib\assert.c:81
0x4008c4a7: xQueueReceive at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/FreeRTOS-Kernel\queue.c:1479
0x40119241: queue_recv_wrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_wifi/esp32\esp_adapter.c:321
0x40105716: wpa2_task at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/wpa_supplicant/esp_supplicant/src\esp_eap_client.c:208

Other Steps to Reproduce

No response

I have checked existing issues, online documentation and the Troubleshooting Guide

• [X] I confirm I have checked existing issues, online documentation and Troubleshooting guide.

[hlwalZ]

Yeah something is busted here but it might not be anything with the internet library itself...

I've been programming my Wemos lolin s2 mini with "ESP32S2 dev module" selected for over a month which is the incorrect board but WiFi worked fine untill this week. If I switch my selected board to "lolin s2 mini" (which is the correct board) inside the Arduino IDE I am able to connect to eduroam without any problems.

Guess I'll be staying with the old core for a while until things are "really stable".

[me-no-dev]

what you are saying does not make much sense :) why would the board model matter? Maybe it was some cache? I would not say that 3.0.0 is more unstable than 2.0.17. Just many things have changed in both IDF and here as a consequence

[hlwalZ]

The board model does matter. I'm backtracking right now to check where ESPS2 dev module stops working. ESP32 core 2.0.17, 3.0.0-alpha, 3.0.0-rc1 connections to eduroam all work fine and starts breaking at 3.0.0-rc2.

Why I'm saying that it doesn't feel stable is because my ESP just dies if it tries connecting to eduroam. I have a fail counter which should allow up to 30 failed attempts but after 2 dots it just completely shuts itself down.

Edit: I'm currently on school and don't have any capabilities to record the behaviour and post it on Youtube here but I can hop on a Discord call and livestream it to you if you don't believe me haha

[hlwalZ]

FYI: 2.0.17, 3.0.0-rc2 and 3.0.0 with lolin s2 mini selected does not break

[me-no-dev]

RC3 and 3.0.0 final use the same IDF and nothing about WiFi has changed.

[hlwalZ]

I never mentioned RC3, the WiFi breaks at RC2

[me-no-dev]

between RC1 and RC2 we have updated IDF a few times to get to 5.1.4. That is it. It still does not explain why would it work when you select another board. The only thing that makes sense is some cache being kept when it should not.

[hlwalZ]

Unfortunately I'm just a freshmen and when we start talking about cache I won't be able to keep up sorry :(

I've been avoiding using lolin s2 mini because of some lightsensor library that doesn't detect a I2C connection when I have it selected and if I use ESP32S2 dev module it can give me readings without any issues.

I'll start looking at a fix for that lightsensor library when I have the skills but since I can't even fix that a.t.m. I won't be of use here. Best I can do is tell you if something does or doesn't work.

Anyway good luck and please give us (OP and me) an update or explanation if you found anything because I'm really interested🙂

Kind regards from Utrecht

[tutotio]

Next Monday I can try it with RC2 and RC3 and give you feedback.

[hlwalZ]

@tutotio please also check RC1, problems only arose for me starting at RC2 while RC1 worked fine. Also on a second note, I'm curious if you too are developing with the wrong board selected like I was doing🤔

[tutotio]

I tested this code with all v3.0 versions and the result was this:

Version 3.0.0: Break Version 3.0.0 RC3: Break Version 3.0.0 RC2: Break Version 3.0.0 RC1: Works fine with the new v3 library: esp_eap_client.h

The problen begins in RC2.

#include <WiFi.h>
#include <HTTPClient.h>
#include <WiFiClientSecure.h>

#if __has_include("esp_eap_client.h")
#include "esp_eap_client.h"
#else
#include "esp_wpa2.h"
#endif

#include <Wire.h>

#define EAP_IDENTITY "xxxxxxxxxxxxxxx@xxxxxxx.xxx"  //if connecting from another corporation, use identity@organization.domain in Eduroam
#define EAP_PASSWORD "xxxxxxxxxxxxx"  //your Eduroam password
const char *ssid = "eduroam";    // Eduroam SSID
esp_eap_ttls_phase2_types ttlsPhase2Type = ESP_EAP_TTLS_PHASE2_PAP;

/** 
 * WPA2 Enterprise CA Certificate for 802.1 User Authentication
 * in eduroam university wifi.
 */
const char* wifiWpa2EnterpriseCACertificate= \
"-----BEGIN CERTIFICATE-----\n" \
"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\n" \
...
"XXXXXXXXXXXXXXXXXXXXXXXX\n" \
"-----END CERTIFICATE-----\n";

int counter = 0;

void setup() {
  Serial.begin(115200);
  delay(1000);
  Serial.println();
  Serial.print("Connecting to network: ");
  Serial.println(ssid);
  WiFi.disconnect(true);  //disconnect form wifi to set new wifi connection
  WiFi.mode(WIFI_STA);    //init wifi mode

#if __has_include("esp_eap_client.h")
    Serial.print("esp32 by Espressif Systems (Versión 3)");
    esp_eap_client_set_ca_cert((unsigned char*)wifiWpa2EnterpriseCACertificate, strlen(wifiWpa2EnterpriseCACertificate) + 1);
    esp_eap_client_set_identity((uint8_t *)EAP_IDENTITY, strlen(EAP_IDENTITY));  //provide identity
    esp_eap_client_set_username((uint8_t *)EAP_IDENTITY, strlen(EAP_IDENTITY));  //provide username
    esp_eap_client_set_password((uint8_t *)EAP_PASSWORD, strlen(EAP_PASSWORD));  //provide password
    esp_eap_client_set_ttls_phase2_method(ttlsPhase2Type);
    esp_wifi_sta_enterprise_enable();
#else
    Serial.print("esp32 by Espressif Systems (Versión 2)");
    esp_wifi_sta_wpa2_ent_set_ca_cert((unsigned char*)wifiWpa2EnterpriseCACertificate, strlen(wifiWpa2EnterpriseCACertificate) + 1);
    esp_wifi_sta_wpa2_ent_set_identity((uint8_t *)EAP_IDENTITY, strlen(EAP_IDENTITY));  //provide identity
    esp_wifi_sta_wpa2_ent_set_username((uint8_t *)EAP_IDENTITY, strlen(EAP_IDENTITY));  //provide username --> identity and username is same
    esp_wifi_sta_wpa2_ent_set_password((uint8_t *)EAP_PASSWORD, strlen(EAP_PASSWORD));  //provide password
    esp_wifi_sta_wpa2_ent_set_ttls_phase2_method(ttlsPhase2Type);
    esp_wifi_sta_wpa2_ent_enable();
#endif

  WiFi.begin(ssid);  //connect to wifi
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
    counter++;
    if (counter >= 60) {  //after 30 seconds timeout - reset board
      ESP.restart();
    }
  }
  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address set: ");
  Serial.println(WiFi.localIP());  //print LAN IP

}

void loop() {
}

[me-no-dev]

That would mean that something in WiFi changed in ESP-IDF and is now causing this to fail. We have also poked our colleagues. Hope to get some info soon

[tutotio]

What parameters should I use in the V3 WiFi.begin(...) call for the connection in the example above without import esp_eap_client.h using ssid, identity, username, password, ca_cert and phase2_method (ESP_EAP_TTLS_PHASE2_PAP)? thanks.

[me-no-dev]

WiFi.begin(ssid, WPA2_AUTH_TTLS, EAP_IDENTITY, EAP_IDENTITY, EAP_PASSWORD, wifiWpa2EnterpriseCACertificate, NULL, NULL, ESP_EAP_TTLS_PHASE2_PAP);

[tutotio]

Using direct method to connect to enterprise networks:

Version 3.0.0: Break Version 3.0.0 RC3: Break Version 3.0.0 RC2: Break Version 3.0.0 RC1: No break, no connection ([ 2478][W][STA.cpp:157] _onStaArduinoEvent(): Reason: 23 - 802_1X_AUTH_FAILED)

#include <WiFi.h>
#include "esp_eap_client.h"

#define EAP_IDENTITY "xxxxxxxxxxxxxxx@xxxxxxx.xxx"  //if connecting from another corporation, use identity@organization.domain in Eduroam
#define EAP_PASSWORD "xxxxxxxxxxxxx"  //your Eduroam password
const char *ssid = "eduroam";    // Eduroam SSID

/** 
 * WPA2 Enterprise CA Certificate for 802.1 User Authentication
 * in eduroam university wifi.
 */
const char* wifiWpa2EnterpriseCACertificate= \
"-----BEGIN CERTIFICATE-----\n" \
"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX\n" \
...
"XXXXXXXXXXXXXXXXXXXXXXXX\n" \
"-----END CERTIFICATE-----\n";

int counter = 0;

void setup() {
  Serial.begin(115200);
  delay(1000);
  Serial.println();
  Serial.print("Connecting to network: ");
  Serial.println(ssid);
  WiFi.disconnect(true);  //disconnect form wifi to set new wifi connection
  WiFi.mode(WIFI_STA);    //init wifi mode

  WiFi.begin(ssid, WPA2_AUTH_TTLS, EAP_IDENTITY, EAP_IDENTITY, EAP_PASSWORD, wifiWpa2EnterpriseCACertificate, NULL, NULL, ESP_EAP_TTLS_PHASE2_PAP);  //connect to wifi
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
    counter++;
    if (counter >= 60) {  //after 30 seconds timeout - reset board
      ESP.restart();
    }
  }
  Serial.println("");
  Serial.println("WiFi connected");
  Serial.println("IP address set: ");
  Serial.println(WiFi.localIP());  //print LAN IP

}

void loop() {
}

[me-no-dev]

RC1 is missing the type, as far as I remember. Probably why you get AUTH FAIL

[VojtechBartoska]

any actions on this issue @me-no-dev?

[me-no-dev]

Please note that from MbedTLS-3.0 onwards, MbedTLS does not support SSL-3.0, TLS-v1.0, TLS-v1.1 versions. Incase your server is using one of these version, it is advisable to update your server.

Does this ring a bell @tutotio ?

[tutotio]

@me-no-dev our wifi is updated. This is not the problem. If it was not updated, the code should give an error, not break and restart the esp32.

[me-no-dev]

any actions on this issue @me-no-dev?

I'm out of ideas, this needs to be looked at by the WiFi team

[tutotio]

I think it breaks before trying to connect, because there is no record left in the university's wifi log when the device breaks.

[tutotio]

The issue remains unresolved in version 3.0.1

[VojtechBartoska]

Adding to 3.0.2 milestone to investigate this.

[VojtechBartoska]

Hello,

we have discussed this issue today with Team and we are not able to test this properly as we do not have access to Eduroam network.

I am labelling this as "Community help needed".

Looks we will not be able to help here.

[tutotio]

how I can help?

[VojtechBartoska]

@tutotio probably someone from Community can help, as I said, we do not have access to Eduroam network.

[Wallmeier]

I've just tested the new version 3.0.2. The problem still exists and is not limited to the Eduroam network. I'm using a wireless network from Unifi with the integrated RADIUS server and WPA 3 Enterprise PEAP-MS-CHAP v2 authentication. With 3.0.0-rc1 I was able to connect to this WiFi-network.

I'm getting a similar call stack:

assert failed: xQueueReceive queue.c:1475 (( pxQueue ))
Backtrace: 0x40083765:0x3ffddeb0 0x4008e4b1:0x3ffdded0 0x40094476:0x3ffddef0 0x4008ebd3:0x3ffde020 0x401912a9:0x3ffde060 0x401883ee:0x3ffde080 0x400913d2:0x3ffde0c0
#0  0x40083765 in panic_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/panic.c:466
#1  0x4008e4b1 in esp_system_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_system/port/esp_system_chip.c:84 
#2  0x40094476 in __assert_func at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/newlib/assert.c:81
#3  0x4008ebd3 in xQueueReceive at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/FreeRTOS-Kernel/queue.c:1479 (discriminator 2)
#4  0x401912a9 in queue_recv_wrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp_wifi/esp32/esp_adapter.c:321   
#5  0x401883ee in wpa2_task at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/wpa_supplicant/esp_supplicant/src/esp_eap_client.c:208
#6  0x400913d2 in vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/FreeRTOS-Kernel/portable/xtensa/port.c:162

[tutotio]

I think the problem may have to do with the storage space where the queue is stored. In the esp-idf version 5.1.2 used in version 3.0.0-rc1 (it works), the queue is created with the method

xQueueCreate(queue_len, item_size);

In version 5.1.4 used by versions 3.0.0-rc2 and later, the queue is created with the method xQueueCreateWithCaps(queue_len, item_size, MALLOC_CAP_SPIRAM); to allocate the queue into SPIRAM.

Debugging the code you get this error:

Info : Set GDB target to 'esp32s3.cpu1' Info : [esp32s3.cpu0] Target halted, PC=0x4200E249, debug_reason=00000000 Info : [esp32s3.cpu0] Halt cause (assert failed: xQueueGenericSend queue.c:872 (pxQueue)) Info : [esp32s3.cpu1] Halt cause (assert failed: xQueueGenericSend queue.c:872 (pxQueue)) Info : [esp32s3.cpu1] Halt cause (29) - (Store prohibited) Info : [esp32s3.cpu1] Halt cause (29) - (Store prohibited) Info : [esp32s3.cpu0] Target halted, PC=0x403772A5, debug_reason=00000001

IMPORTANT: If I enable OPI PSRAM in ESP32 S3 board the code works and can connect to eduroam with framework 3.0.2 with esp-idf ver 5.1.4.

I think the problem is that the framework always tries to create the queue in the PSRAM memory even though the chip does not have it available, and when trying to write to the memory it fails and restarts the device.

[me-no-dev]

@tutotio very interesting! can you point to the lines before and after, so that we can report it?

[tutotio]

/esp-idf/components/esp_wifi/esp32/esp_adapter.c (ver 5.1.2)

static void queue_create_wrapper(uint32_t queue_len, uint32_t item_size) { return (void )xQueueCreate(queue_len, item_size); }

/esp-idf/components/esp_wifi/esp32/esp_adapter.c (ver 5.1.4)

static void * queue_create_wrapper(uint32_t queue_len, uint32_t item_size) {

if ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(5, 1, 0)

/*
 * Since release/v5.1, FreeRTOS has been updated to always use internal memory (i.e., DRAM)
 * for dynamic memory allocation. Calling FreeRTOS creation functions (e.g., xTaskCreate(), xQueueCreate())
 * will guarantee that the memory allocated for those tasks/objects is from internal memory.
 * For more details, please refer to the Migration Guide in release/v5.1.
 */

if CONFIG_SPIRAM_USE_MALLOC

/* Use xQueueCreateWithCaps() to allocate from SPIRAM */
return (void *)xQueueCreateWithCaps(queue_len, item_size, MALLOC_CAP_SPIRAM);

else

return (void *)xQueueCreate(queue_len, item_size);

endif

else

return (void *)xQueueCreate(queue_len, item_size);

endif

}

Maybe CONFIG_SPIRAM_USE_MALLOC is not defined properly.

[Wallmeier]

If I take a look at the file esp32/sdkconfig, I can find the following line: CONFIG_SPIRAM_USE_MALLOC=y

With @tutotio's oberservation, this should mean that the libs are compiled with CONFIG_SPIRAM_USE_MALLOC define, but will only work correctly on boards with PSRAM.

Is it possible for you to provide a libs archive where the sdkconfig does not contain CONFIG_SPIRAM_USE_MALLOC to confirm the assumption?

[Wallmeier]

I just used the esp32-arduino-lib-builder library builder to build my own libraries. I was not able to disable only CONFIG_SPIRAM_USE_MALLOC, but by disabling CONFIG_SPIRAM completely, the WiFi worked fine again (connecting to WPA3 PEAP MSCHAPv2 network).

[me-no-dev]

We have submitted issue report about similar issue to the WiFi team, but there is no response yet. It seems that they think that if PSRAM is enabled, then it is also started/connected, which is not the case in Arduino. Hopefully we will get some traction soon.

[NicoWallmeier]

@me-no-dev Is there any progress? Is there a ticket for this in the esp32 idf project?

[me-no-dev]

@Wallmeier there is an effort to fix this in IDF. It will take some time to trickle down the IDF versions. First will probably come to the 3.1.x Arduino

[tutotio]

I have tried again to migrate my code from version 2 to version 3 and I have realized that the problem with the esp32 restarts has been resolved in version 3.0.5 (Great!!!)

Now my connection works fine with this code:

  WiFi.mode(WIFI_STA);  //init wifi mode
  esp_eap_client_set_ca_cert((uint8_t *)wifiWpa2EnterpriseCACertificate, strlen(wifiWpa2EnterpriseCACertificate) +1);
  esp_eap_client_set_identity((uint8_t *)EAP_IDENTITY, strlen(EAP_IDENTITY));  //provide identity
  esp_eap_client_set_username((uint8_t *)EAP_IDENTITY, strlen(EAP_IDENTITY));  //provide username
  esp_eap_client_set_password((uint8_t *)EAP_PASSWORD, strlen(EAP_PASSWORD));  //provide password
  esp_eap_client_set_ttls_phase2_method(ESP_EAP_TTLS_PHASE2_PAP);
  esp_wifi_sta_enterprise_enable();
  WiFi.begin(ssid);  //connect to wifi

Solved this problem, now I want to use direct connection without include esp_eap_client.h library. WiFi.begin(ssid, WPA2_AUTH_TTLS, EAP_IDENTITY, EAP_IDENTITY, EAP_PASSWORD, wifiWpa2EnterpriseCACertificate, NULL, NULL, ESP_EAP_TTLS_PHASE2_PAP);

I found two problems with it:

1- ESP_EAP_TTLS_PHASE2_PAP variable is not accessible from the WiFi library, so I have to mandatory add the code #include "esp_eap_client.h" on my library.

2- The ca_cert fails and don't connect. I think the problem is caused by the strlen() function does return the length of the certificate without the terminating 0x00. The esp_eap_client_set_ca_cert() does need the terminating 0x00, so, i think you must add +1 to strlen() here: https://github.com/espressif/arduino-esp32/blob/master/libraries/WiFi/src/STA.cpp#L484 esp_eap_client_set_ca_cert((uint8_t *)ca_pem, strlen(ca_pem)+1);

[Wallmeier]

With Arduino Core 3.0.7 and 3.1.0-rc2 the WiFi connection problem with WPA 3 Enterprise PEAP-MS-CHAP v2 authentication is solved for me.