ESP32 - Individual enrollment works, Group enrollment fails - Root and Intermediate Cert uploaded to DPS correctly

[sckulkarni246]

Please provide us with the following information:

This issue is for a: (mark with an x)

- [x] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Working case - Individual Enrolment

• Upload root certificate to DPS
• Create an individual enrolment policy with the device certificate correctly uploaded
• Provide the device key and certificate correctly in the ESP32 sample example
• Enable DPS support in the menuconfig of the ESP32 project and supply the correct idscope and the registrationId
• Try out the example - it works and able to do MQTT

Non-working case - Group Enrolment

• Delete the individual enrolment policy
• Create a group enrolment policy with the correct intermediate certificate uploaded
• Remove the device from IoT Hub and try the example again - it does not work, logs are below

Any log messages given by the failure

I (578) cpu_start: Starting scheduler on PRO CPU.
I (0) cpu_start: Starting scheduler on APP CPU.
I (777) wifi:wifi driver task: 3ffaf6c8, prio:23, stack:6656, core=0
I (777) system_api: Base MAC address is not set
I (777) system_api: read default base MAC address from EFUSE
I (797) wifi:wifi firmware version: 7679c42
I (807) wifi:wifi certification version: v7.0
I (807) wifi:config NVS flash: enabled
I (807) wifi:config nano formating: disabled
I (807) wifi:Init data frame dynamic rx buffer num: 32
I (807) wifi:Init management frame dynamic rx buffer num: 32
I (817) wifi:Init management short buffer num: 32
I (817) wifi:Init dynamic tx buffer num: 32
I (827) wifi:Init static rx buffer size: 1600
I (827) wifi:Init static rx buffer num: 10
I (837) wifi:Init dynamic rx buffer num: 32
I (837) wifi_init: rx ba win: 6
I (837) wifi_init: tcpip mbox: 32
I (847) wifi_init: udp mbox: 6
I (847) wifi_init: tcp mbox: 6
I (857) wifi_init: tcp tx win: 5744
I (857) wifi_init: tcp rx win: 5744
I (857) wifi_init: tcp mss: 1440
I (867) wifi_init: WiFi IRAM OP enabled
I (867) wifi_init: WiFi RX IRAM OP enabled
I (877) sample_azureiot: Connecting to Shashank Kulkarni...
I (877) phy_init: phy_version 4670,719f9f6,Feb 18 2021,17:07:07
I (987) wifi:mode : sta (7c:9e:bd:ed:11:30)
I (997) wifi:enable tsf
I (997) sample_azureiot: Waiting for IP(s)
I (3047) wifi:new:<6,0>, old:<1,0>, ap:<255,255>, sta:<6,0>, prof:1
I (3807) wifi:state: init -> auth (b0)
I (3817) wifi:state: auth -> assoc (0)
I (3827) wifi:state: assoc -> run (10)
I (3847) wifi:connected with Shashank Kulkarni, aid = 1, channel 6, BW20, bssid = de:d9:6a:a9:d3:dc
I (3857) wifi:security: WPA2-PSK, phy: bgn, rssi: -32
I (3857) wifi:pm start, type: 1

I (3897) wifi:AP's beacon interval = 102400 us, DTIM period = 1
I (4667) esp_netif_handlers: sample_azureiot: sta ip: 172.20.10.8, mask: 255.255.255.240, gw: 172.20.10.1
I (4667) sample_azureiot: Got IPv4 event: Interface "sample_azureiot: sta" address: 172.20.10.8
I (4677) sample_azureiot: Connected to sample_azureiot: sta
I (4677) sample_azureiot: - IPv4 address: 172.20.10.8
I (4687) sample_azureiot: Waiting for time synchronization with SNTP server
W (6567) wifi:<ba-add>idx:0 (ifx:0, de:d9:6a:a9:d3:dc), tid:0, ssn:2, winSize:64
I (6637) sample_azureiot: Notification of a time synchronization event
I (6707) AZ IOT: Creating a TLS connection to global.azure-devices-provisioning.net:8883.

I (6957) esp-tls-mbedtls: Initialize the ATECC interface...
I (10197) tls_freertos: (Network connection 0x3ffca850) Connection to global.azure-devices-provisioning.net established.
I (10367) MQTT: Packet received. ReceivedBytes=2.
I (10367) MQTT: CONNACK session present bit not set.
I (10367) MQTT: Connection accepted.
I (10377) MQTT: Received MQTT CONNACK successfully from broker.
I (10387) MQTT: MQTT connection established with the broker.
I (10387) AZ IOT: AzureIoTProvisioning established an MQTT connection with global.azure-devices-provisioning.net
I (12407) AZ IOT: AzureIoTProvisioning attempting to subscribe to the MQTT topic: devices/+/messages/devicebound/#
I (12537) MQTT: Packet received. ReceivedBytes=3.
I (14987) MQTT: Packet received. ReceivedBytes=192.
I (14987) MQTT: De-serialized incoming PUBLISH packet: DeserializerResult=MQTTSuccess.
I (14987) MQTT: State record updated. New state=MQTTPublishDone.
Guru Meditation Error: Core  1 panic'ed (StoreProhibited). Exception was unhandled.

Core  1 register dump:
PC      : 0x40103543  PS      : 0x00060b30  A0      : 0x80103b1e  A1      : 0x3ffca6b0  
0x40103543: capture_tcp_transport_error at /home/shashank/work/git/espressif/esp-idf/components/tcp_transport/transport.c:332

A2      : 0x3ffaf85c  A3      : 0x00008008  A4      : 0x00000001  A5      : 0x3ffca560  
A6      : 0x0000004c  A7      : 0x3ffca560  A8      : 0x80103514  A9      : 0x3ffca690  
A10     : 0x00000000  A11     : 0xffff8d80  A12     : 0x00000001  A13     : 0x3ffbcd40  
A14     : 0x00000001  A15     : 0xff000000  SAR     : 0x0000000a  EXCCAUSE: 0x0000001d  
EXCVADDR: 0x00000000  LBEG    : 0x4000c2e0  LEND    : 0x4000c2f6  LCOUNT  : 0xffffffff  

Backtrace:0x40103540:0x3ffca6b00x40103b1b:0x3ffca6d0 0x4016945d:0x3ffca700 0x400d8a65:0x3ffca720 0x400dcf79:0x3ffca750 0x400dbcb2:0x3ffca770 0x400dbdba:0x3ffca7a0 0x400dab99:0x3ffca7c0 0x400da79c:0x3ffca7e0 0x400da9e4:0x3ffca810 0x400d7db2:0x3ffca830 0x400d7ecb:0x3ffca8b0 0x4008ce65:0x3ffca960 
0x40103540: capture_tcp_transport_error at /home/shashank/work/git/espressif/esp-idf/components/tcp_transport/transport.c:332

0x40103b1b: ssl_read at /home/shashank/work/git/espressif/esp-idf/components/tcp_transport/transport_ssl.c:249

0x4016945d: esp_transport_read at /home/shashank/work/git/espressif/esp-idf/components/tcp_transport/transport.c:190

0x400d8a65: TLS_Socket_Recv at /home/shashank/work/git/espressif/sck-stuff/azure-stuff/iot-middleware-freertos-samples/demos/projects/ESPRESSIF/esp32/build/../components/sample-azure-iot/transport_tls_esp32.c:168

0x400dcf79: MQTT_GetIncomingPacketTypeAndLength at /home/shashank/work/git/espressif/sck-stuff/azure-stuff/iot-middleware-freertos-samples/libs/azure-iot-middleware-freertos/libraries/coreMQTT/source/core_mqtt_serializer.c:2359

0x400dbcb2: receiveSingleIteration at /home/shashank/work/git/espressif/sck-stuff/azure-stuff/iot-middleware-freertos-samples/libs/azure-iot-middleware-freertos/libraries/coreMQTT/source/core_mqtt.c:1297

0x400dbdba: MQTT_ProcessLoop at /home/shashank/work/git/espressif/sck-stuff/azure-stuff/iot-middleware-freertos-samples/libs/azure-iot-middleware-freertos/libraries/coreMQTT/source/core_mqtt.c:2185

0x400dab99: AzureIoTMQTT_ProcessLoop at /home/shashank/work/git/espressif/sck-stuff/azure-stuff/iot-middleware-freertos-samples/libs/azure-iot-middleware-freertos/ports/coreMQTT/azure_iot_core_mqtt.c:182

0x400da79c: prvProvClientRunWorkflow at /home/shashank/work/git/espressif/sck-stuff/azure-stuff/iot-middleware-freertos-samples/libs/azure-iot-middleware-freertos/source/azure_iot_provisioning_client.c:537

0x400da9e4: AzureIoTProvisioningClient_Register at /home/shashank/work/git/espressif/sck-stuff/azure-stuff/iot-middleware-freertos-samples/libs/azure-iot-middleware-freertos/source/azure_iot_provisioning_client.c:902

0x400d7db2: prvIoTHubInfoGet at /home/shashank/work/git/espressif/sck-stuff/azure-stuff/iot-middleware-freertos-samples/demos/sample_azure_iot_pnp/sample_azure_iot_pnp.c:590 (discriminator 1)

0x400d7ecb: prvAzureDemoTask at /home/shashank/work/git/espressif/sck-stuff/azure-stuff/iot-middleware-freertos-samples/demos/sample_azure_iot_pnp/sample_azure_iot_pnp.c:359

0x4008ce65: vPortTaskWrapper at /home/shashank/work/git/espressif/esp-idf/components/freertos/port/xtensa/port.c:131

Expected/desired behavior

• Group enrolment should also work, we should see MQTT happening correctly

OS and Version?

• Ubuntu 20.04 LTS
• ESP IDF v4.4

Versions

My repo's HEAD is at bcfbaab143e5ece442b5d0d45712f2f681c3b237

Mention any other details that might be useful

Nothing to report. Look forward to a resolution!

[sckulkarni246]

Hello team,

I finally figured out the issue. It was to do with the fact that ESP32 software does not send the intermediate certificate during the TLS handshake.

The hint came from here: https://github.com/Azure/azure-iot-sdk-csharp/issues/1010

Specifically, this suggestion:

You could chose one of the following two options :

Upload intermediate only and do Proof of possession on that or Include intermediate in tls handshake Any of the above two options can be valid options.

I chose the first work-around i.e. upload intermediate and do p-o-p on it. Thanks!

Regards, Shashank