espressif / esp-idf

Espressif IoT Development Framework. Official development framework for Espressif SoCs.
Apache License 2.0
13.91k stars 7.33k forks source link

IDF 5.0 cannot connect to Google MQTT server (IDFGH-8568) #10018

Closed Nick-tecpal closed 1 year ago

Nick-tecpal commented 2 years ago

Answers checklist.

IDF version.

v5.0-beta1-742-g7bd5af7f1e-dirt

Operating System used.

Linux

How did you build your project?

If you are using Windows, please specify command line type.

Development Kit.

ESP-S3-DevkitC-1 V1.1

Power Supply used.

USB

What is the expected behavior?

IDF 5.0 can connect to our Google MQTT server.

What is the actual behavior?

Our server return 34 ,  /* @cond Internal. Numeric code: 34 / IOTC_MQTT_NOT_AUTHORIZED, /* @endcond /

Steps to reproduce.

ccc8b383-499e-4b4c-9def-63e975ed8ca6 85da900a-4ec2-4a49-a909-1891727096d3

Debug Logs.

I (12144) Tecpal net decode: [Heap: 5898836]
upgrade notify app net result -> ({"uri":"/wifi status",
. "seg": 2, "result": 4, "params": ("ssid": "TGI-WiFi-VPN", "rssi": -58, "ipAddr": "192.168.50.19"}}]
I (12654) Tepal Giot fml: [Heap: 5892036]mqtt private key [-----BEGIN EC PRIVATE KEY ---
-----END EC PRIVATE KEY- ----]
I (12664) Tepal Giot fml: (Heap: 5891888]matt connect info proiects/tecpal-dev/locations/asia-eastl/registries/cp-iot-registry-asia-sit/devices/prA1F362661
E
(14354) Tecpal Giot fml: [Heap:5891036][../components/tecpal sdk/FML/module/tecpal sdk giot.c:142]->on connection state changed cbt}
MOTT Open Failed with reason[34].
I
(14354) Tepal Giot fml:
[Heap: 5891084]
iot core ready to closed
E (14364) Tecpal Giot
fml.
[Hean:5892274
tecpal
sdk/FML/module/tecpal sdk aiot.c:2291->iot matt
task handlerf1 • Tor Core Delete fol..

More Information.

No response

euripedesrocha commented 2 years ago

Hi @Nick-tecpal, could you clarify on which library you are using for MQTT client?

Jiven-tecpal commented 2 years ago

The tracking discovery is caused by IDF5.0 using the latest mbedtls library,Thus causing the JWT token signature exception

Jiven-tecpal commented 2 years ago

google iot connect info: client id:projects/tecpal-dev/locations/asia-east1/registries/cp-iot-registry-asia-sit/devices/PC_A1E3626E jwt:eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE2NjcyNzY3OTcsImV4cCI6MTY2NzMxOTk5NywiYXVkIjoidGVjcGFsLWRldiJ9.AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==

Jiven-tecpal commented 2 years ago

@euripedesrocha please help

Jiven-tecpal commented 2 years ago

google iot repo commit id:758425b9c2ec82d35a0af77e31dcd9b69c303f8a (HEAD -> master, origin/master, origin/HEAD) @euripedesrocha

euripedesrocha commented 2 years ago

Hi @jiven57316 is this related to @Nick-tecpal original issue? If not, could you please open a new issue describing your problem.

Jiven-tecpal commented 2 years ago

“Hi @jiven57316 is this related to @Nick-tecpal original issue? If not, could you please open a new issue describing your problem.”-----Yes,We are in the same company and i working on this issue. @euripedesrocha

Nick-tecpal commented 2 years ago

Hi @euripedesrocha , we're trying to figure out the root cause. It seems the iotc_bsp_ecc function is not working in IDF 5.0 which is using the latest mbedtls library. WechatIMG551

zztiger123 commented 2 years ago

update: We evaluated this issue further and we observed that crypto and TLS layers from Google IoT SDK would need careful migration to mbedTLS-3.x. Please find migration guide available here. Please file an issue on upstream repository if you need further help on this topic. From IDF side, we do not see any issue as such and hence we would close it. Thank you.

mahavirj commented 1 year ago

Since this change is from the external component as highlighted in the earlier comment, closing it from IDF side.

Nick-tecpal commented 1 year ago

Hi, we have found a new issue related to the mbedtls V3.2.1. 7201675220811_ pic_hd 7211675220812_ pic_hd

Nick-tecpal commented 1 year ago

issue:When switching to IDF5.0, the JWT generated before the MQTT connection fails to generate a valid signature, so the value of AAAAA appears and an error code of 34 is returned.

chip:ESP32S3N8R8

idf-version:885e501(HEAD -> v5.0, origin/release/v5.0)

mbedtls version:7b428b1bf260ce1fec4e3ffb6494070d439a3a67 components/mbedtls/mbedtls (mbedtls-3.2.1-6-g7b428b1bf)

google mqtt info:

I (10263) Tecpal_Giot_fml: [Heap:7620888]mqtt connect info [projects/tecpal-dev/locations/asia-east1/registries/cp-iot-registry-asia-sit/devices/PC_A1E3626E],JWT[eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE2NzU4Mzg3NTcsImV4cCI6MTY3NTg4MTk1NywiYXVkIjoidGVjcGFsLWRldiJ9.AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==],state:0

Correct jwt token:

"eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE2NjcyODUwMzYsImV4cCI6MTY2NzMyODIzNiwiYXVkIjoidGVjcGFsLWRldiJ9.NmmMPO9_5VBC7adsWCYAFk8-Vq2Z6tw2sAsd-PVeWCNpXIGZ361buLUWEafe6ZoDnNzhwyroPGCkD_yzc4LTiA=="

Actual jwt token:

"eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE2NzU4Mzg3NTcsImV4cCI6MTY3NTg4MTk1NywiYXVkIjoidGVjcGFsLWRldiJ9.AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=="

Related source code from application layer:

static void iot_mqtt_task_handler(void pvParameters) { security_key_t security_key = tecpal_sdk_system_get_security_key(); iotc_crypto_key_data_t iotc_connect_private_key_data; iotc_connect_private_key_data.crypto_key_signature_algorithm = IOTC_CRYPTO_KEY_SIGNATURE_ALGORITHM_ES256; iotc_connect_private_key_data.crypto_key_union_type = IOTC_CRYPTO_KEY_UNION_TYPE_PEM; //iotc_connect_private_key_data.crypto_key_union.key_pem.key = (char )private_key; iotc_connect_private_key_data.crypto_key_union.key_pem.key = (char )security_key->mqtts_ca_cert.pdata.data;

iotc_state_t iot_state; 
iot_state = iotc_initialize(); 
if (IOTC_STATE_OK != iot_state) { 
    GIOT_FML_LOGE(" iotc failed to initialize, error: %d\n", iot_state); 
    is_iot_open = false; 
    iotc_shutdown(); 
    vTaskDelete(mqtt_task_handle); 
} 
iotc_handle = iotc_create_context(); 
if (IOTC_INVALID_CONTEXT_HANDLE >= iotc_handle) { 
    GIOT_FML_LOGE(" iotc failed to create context, error: %d\n", -iotc_handle); 
    iotc_shutdown(); 

    is_iot_open = false; 
    vTaskDelete(mqtt_task_handle); 
}

/ Generate the client authentication JWT, which will serve as the MQTT password. / size_t bytes_written = 0; iot_state = iotc_create_iotcore_jwt( CONFIG_GIOT_PROJECT_ID, /jwt_expiration_period_sec=/12*3600, &iotc_connect_private_key_data, jwt, IOTC_JWT_SIZE, &bytes_written); GIOT_FML_LOGI("mqtt private key [%s],jwt[%s],len:%d",iotc_connect_private_key_data.crypto_key_union.key_pem.key,jwt,IOTC_JWT_SIZE); //strcpy(jwt,"eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE2NjcyODUwMzYsImV4cCI6MTY2NzMyODIzNiwiYXVkIjoidGVjcGFsLWRldiJ9.NmmMPO9_5VBC7adsWCYAFk8-Vq2Z6tw2sAsd-PVeWCNpXIGZ361buLUWEafe6ZoDnNzhwyroPGCkD_yzc4LTiA==");
//use this jwt can solve the issue if (IOTC_STATE_OK != iot_state) { GIOT_FML_LOGE("iotc_create_iotcore_jwt returned with error: %ul", iot_state); // sdk_event_center_notify_evt(NTP_TIME_SYNC,NULL,0); is_iot_open = false; iotc_delete_context(iotc_handle); iotc_handle = IOTC_INVALID_CONTEXT_HANDLE; vTaskDelete(mqtt_task_handle); }

iot_state = iot_mqtt_connect(); 
if (IOTC_STATE_OK != iot_state) { 
    GIOT_FML_LOGE("iotc_connect fail with ret:[%d]", iot_state); 
    vTaskDelete(mqtt_task_handle); 
} 
iotc_events_process_blocking(); 
GIOT_FML_LOGI("\r\n \t iot core ready to closed ............................"); 

if(mqtt_task_handle){ 
    iotc_delete_context(iotc_handle); 
    iotc_shutdown(); 
    iotc_handle = IOTC_INVALID_CONTEXT_HANDLE; 
    GIOT_FML_LOGE("IOT Core Delete [%d]....\r\n",is_reconnect_fail); 
    is_iot_open = false; 
    mqtt_task_handle = NULL; 
    if(is_reconnect_fail){ 
        is_reconnect_fail = false; 
        sdk_event_center_notify_evt(NTP_TIME_SYNC ,NULL,0); 
    } 
    vTaskDelete(mqtt_task_handle); 
} 

}

// Related driven code from Google IOT: // create the JWT: b64(h).b64(p).b64(ecc(sha256(b64(h).b64(p)))) // h = header // p = payload // b64 = base64 // sha = Secure Hash Algorithm // ecc = Elliptic Curve Cryptography iotc_state_t iotc_create_iotcore_jwt( const char project_id, uint32_t expiration_period_sec, const iotc_crypto_key_data_t private_key_data, char dst_jwt_buf, size_t dst_jwt_buf_len, size_t bytes_written) { if (NULL == project_id || NULL == private_key_data || NULL == dst_jwt_buf || NULL == bytes_written) { return IOTC_INVALID_PARAMETER; }

if (IOTC_CRYPTO_KEY_SIGNATURE_ALGORITHM_ES256 != private_key_data->crypto_key_signature_algorithm) { return IOTC_ALG_NOT_SUPPORTED_ERROR; }

switch (private_key_data->crypto_key_union_type) { case IOTC_CRYPTO_KEY_UNION_TYPE_PEM: if (NULL == private_key_data->crypto_key_union.key_pem.key) { return IOTC_NULL_KEY_DATA_ERROR; } break; case IOTC_CRYPTO_KEY_UNION_TYPE_SLOT_ID: case IOTC_CRYPTO_KEY_UNION_TYPE_CUSTOM: / it's a valid scenario that the custom data could be null, if the key is hard coded in the BSP / break; default: return IOTC_NOT_IMPLEMENTED; }

if (IOTC_JWT_PROJECTID_MAX_LEN < strlen(project_id)) { *bytes_written = IOTC_JWT_PROJECTID_MAX_LEN; return IOTC_JWT_PROJECTID_TOO_LONG_ERROR; }

iotc_bsp_crypto_state_t ret = IOTC_BSP_CRYPTO_ERROR;

// create base64 encoded header and payload: b64(h).b64(p) IOTC_CHECK_CRYPTO(ret = _iotc_create_iotcore_jwt_b64h_b64p( (unsigned char*)dst_jwt_buf, dst_jwt_buf_len, bytes_written, project_id, expiration_period_sec, "ES256"));

// create sha256 hash of b64(h).b64(p): sha256(b64(h).b64(p)) uint8_t sha256_b64h_b64p[32] = {0}; IOTC_CHECK_CRYPTO(ret = iotc_bsp_sha256(sha256_b64h_b64p, (const uint8_t)dst_jwt_buf, bytes_written));

// add second dot, separating b64(h).b64(p) and b64(eccsignature) dst_jwt_buf[bytes_written] = '.'; ++bytes_written;

// create ecc signature: ecc(sha256(b64(h).b64(p))) size_t bytes_written_ecc_signature = 0; uint8_t ecc_signature[IOTC_JWT_MAX_SIGNATURE_SIZE] = {0}; IOTC_CHECK_CRYPTO(ret = iotc_bsp_ecc(private_key_data, ecc_signature, IOTC_JWT_MAX_SIGNATURE_SIZE, &bytes_written_ecc_signature, sha256_b64h_b64p, 32));

// base64 encode the ecc signature size_t bytes_written_ecc_signature_base64 = 0; ret = iotc_bsp_base64_encode_urlsafe( (unsigned char)dst_jwt_buf + bytes_written, dst_jwt_buf_len - *bytes_written, &bytes_written_ecc_signature_base64, ecc_signature, bytes_written_ecc_signature);

*bytes_written += bytes_written_ecc_signature_base64;

IOTC_CHECK_CRYPTO(ret);

return IOTC_STATE_OK;

err_handling:

switch (ret) { case IOTC_BSP_CRYPTO_STATE_OK: return IOTC_STATE_OK; case IOTC_BSP_CRYPTO_BUFFER_TOO_SMALL_ERROR: return IOTC_BUFFER_TOO_SMALL_ERROR; case IOTC_BSP_CRYPTO_INVALID_INPUT_PARAMETER_ERROR: return IOTC_INVALID_PARAMETER; default: return IOTC_JWT_FORMATTION_ERROR; }

return IOTC_JWT_FORMATTION_ERROR; }

Nick-tecpal commented 1 year ago

Archive.zip

Nick-tecpal commented 1 year ago

MQTT issue on IDF5.0.docx