zhichunlee commented 3 months ago

Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.

ESP-IDF v5.2 release version

Espressif SoC revision.

ESP32(revision v1.0)

Operating System used.

Linux

How did you build your project?

Command line with idf.py

If you are using Windows, please specify command line type.

None

Development Kit.

ESP32_DevKitc_V4

Power Supply used.

USB

What is the expected behavior?

I used smtp client example connect to gmail server with TLS v1.3

What is the actual behavior?

The mbedtls_ssl_handshake() returned -0x7b00, and mbedtls_ssl_read failed with error -0x7b00 form Debug Logs.

Steps to reproduce.

The root cert used as example.
SERVER_USES_STARTSSL was enable same as example.
TLS 1.3 and others setting was configurated as follows: CONFIG_MBEDTLS_SSL_PROTO_TLS1_3=y CONFIG_MBEDTLS_SSL_TLS1_3_COMPATIBILITY_MODE=y CONFIG_MBEDTLS_SSL_TLS1_3_KEXM_PSK=y CONFIG_MBEDTLS_SSL_TLS1_3_KEXM_EPHEMERAL=y CONFIG_MBEDTLS_SSL_TLS1_3_KEXM_PSK_EPHEMERAL=y

CONFIG_MBEDTLS_SSL_VARIABLE_BUFFER_LENGTH=y CONFIG_MBEDTLS_X509_TRUSTED_CERT_CALLBACK=y CONFIG_MBEDTLS_SSL_CONTEXT_SERIALIZATION=y CONFIG_MBEDTLS_SSL_KEEP_PEER_CERTIFICATE=y CONFIG_MBEDTLS_PKCS7_C=y CONFIG_MBEDTLS_SSL_CID_PADDING_GRANULARITY=16

CONFIG_MBEDTLS_SSL_RENEGOTIATION=y CONFIG_MBEDTLS_SSL_PROTO_TLS1_2=y

CONFIG_MBEDTLS_SSL_PROTO_GMTSSL1_1 is not set

CONFIG_MBEDTLS_SSL_PROTO_DTLS is not set

CONFIG_MBEDTLS_SSL_ALPN=y CONFIG_MBEDTLS_CLIENT_SSL_SESSION_TICKETS=y CONFIG_MBEDTLS_SERVER_SSL_SESSION_TICKETS=y

Built and run the example.
The connection and certificate verifying was OK, but at authentication after event MBEDTLS_SSL_TLS1_3_NEW_SESSION_TICKET a new session ticket was untreated. Is there any problem with my settings? or a software bug?

Debug Logs.

I (4567) smtp_example: Seeding the random number generator
I (4577) smtp_example: Loading the CA root certificate...
I (4587) smtp_example: Setting hostname for TLS session...
I (4587) smtp_example: Setting up the SSL/TLS structure...
I (4587) smtp_example: Connecting to smtp.googlemail.com:587...
I (4597) main_task: Returned from app_main()
I (5777) smtp_example: Connected.

220 smtp.googlemail.com ESMTP gm13-20020a17090b100d00b0029beec8e86csm4193529pjb.36 - gsmtp
I (8977) smtp_example: Writing EHLO to server...

250-smtp.googlemail.com at your service, [123.185.111.62]
250-SIZE 35882577
250-8BITMIME
250-STARTTLS
250-ENHANCEDSTATUSCOD
ES
250-PIPELINING
250-CHUNKING
250 SMTPUTF8
I (9387) smtp_example: Writing STARTTLS to server...

220 2.0.0 Ready to start TLS
I (9787) smtp_example: Performing the SSL/TLS handshake...
I (9787) mbedtls: ssl_tls.c:3931 => handshake

I (9797) mbedtls: ssl_msg.c:2370 => flush output

I (9797) mbedtls: ssl_msg.c:2379 <= flush output

I (9807) mbedtls: ssl_tls.c:3850 client state: MBEDTLS_SSL_HELLO_REQUEST

I (9817) mbedtls: ssl_msg.c:2370 => flush output

I (9817) mbedtls: ssl_msg.c:2379 <= flush output

I (9827) mbedtls: ssl_tls.c:3850 client state: MBEDTLS_SSL_CLIENT_HELLO

I (9827) mbedtls: ssl_client.c:938 => write client hello

W (9847) mbedtls: ssl_tls13_generic.c:1580 Perform PSA-based ECDH/FFDH computation.

I (10197) mbedtls: ssl_msg.c:2800 => write handshake message

I (10197) mbedtls: ssl_msg.c:2960 => write record

I (10197) mbedtls: ssl_msg.c:3097 <= write record

I (10207) mbedtls: ssl_msg.c:2921 <= write handshake message

I (10207) mbedtls: ssl_client.c:1026 <= write client hello

I (10217) mbedtls: ssl_msg.c:2370 => flush output

I (10227) mbedtls: ssl_msg.c:2384 message length: 356, out_left: 356

I (10227) mbedtls: ssl_msg.c:2391 ssl->f_send() returned 356 (-0xfffffe9c)

I (10237) mbedtls: ssl_msg.c:2418 <= flush output

I (10247) mbedtls: ssl_tls.c:3850 client state: MBEDTLS_SSL_SERVER_HELLO

I (10247) mbedtls: ssl_tls13_client.c:2033 => ssl_tls13_process_server_hello

I (10257) mbedtls: ssl_msg.c:4134 => read record

I (10267) mbedtls: ssl_msg.c:2172 => fetch input

I (10267) mbedtls: ssl_msg.c:2312 in_left: 0, nb_want: 5

I (10507) mbedtls: ssl_msg.c:2332 in_left: 0, nb_want: 5

I (10517) mbedtls: ssl_msg.c:2335 ssl->f_recv(_timeout)() returned 5 (-0xfffffffb)

I (10517) mbedtls: ssl_msg.c:2357 <= fetch input

I (10527) mbedtls: ssl_msg.c:2172 => fetch input

I (10527) mbedtls: ssl_msg.c:2312 in_left: 5, nb_want: 127

I (10537) mbedtls: ssl_msg.c:2332 in_left: 5, nb_want: 127

I (10537) mbedtls: ssl_msg.c:2335 ssl->f_recv(_timeout)() returned 122 (-0xffffff86)

I (10547) mbedtls: ssl_msg.c:2357 <= fetch input

I (10557) mbedtls: ssl_msg.c:4206 <= read record

I (10557) mbedtls: ssl_tls13_client.c:1508 received ServerHello message

I (10567) mbedtls: ssl_tls13_client.c:509 DHE group name: x25519

I (10757) mbedtls: ssl_tls13_keys.c:1365 => ssl_tls13_generate_handshake_keys

I (10767) mbedtls: ssl_tls13_keys.c:1454 <= ssl_tls13_generate_handshake_keys

W (10767) mbedtls: ssl_tls13_client.c:1985 Switch to handshake keys for inbound traffic

I (10777) mbedtls: ssl_tls13_client.c:2078 <= ssl_tls13_process_server_hello ( ServerHello )

I (10787) mbedtls: ssl_msg.c:2370 => flush output

I (10787) mbedtls: ssl_msg.c:2379 <= flush output

I (10797) mbedtls: ssl_tls.c:3850 client state: MBEDTLS_SSL_ENCRYPTED_EXTENSIONS

I (10807) mbedtls: ssl_tls13_client.c:2215 => parse encrypted extensions

I (10807) mbedtls: ssl_msg.c:4134 => read record

I (10817) mbedtls: ssl_msg.c:2172 => fetch input

I (10817) mbedtls: ssl_msg.c:2312 in_left: 0, nb_want: 5

I (10827) mbedtls: ssl_msg.c:2332 in_left: 0, nb_want: 5

I (10837) mbedtls: ssl_msg.c:2335 ssl->f_recv(_timeout)() returned 5 (-0xfffffffb)

I (10837) mbedtls: ssl_msg.c:2357 <= fetch input

I (10847) mbedtls: ssl_msg.c:2172 => fetch input

I (10857) mbedtls: ssl_msg.c:2312 in_left: 5, nb_want: 6

I (10857) mbedtls: ssl_msg.c:2332 in_left: 5, nb_want: 6

I (10867) mbedtls: ssl_msg.c:2335 ssl->f_recv(_timeout)() returned 1 (-0xffffffff)

I (10877) mbedtls: ssl_msg.c:2357 <= fetch input

W (10877) mbedtls: ssl_msg.c:5017 Ignore ChangeCipherSpec in TLS 1.3 compatibility mode

I (10887) mbedtls: ssl_msg.c:2172 => fetch input

I (10897) mbedtls: ssl_msg.c:2312 in_left: 0, nb_want: 5

I (10897) mbedtls: ssl_msg.c:2332 in_left: 0, nb_want: 5

I (10907) mbedtls: ssl_msg.c:2335 ssl->f_recv(_timeout)() returned 5 (-0xfffffffb)

I (10917) mbedtls: ssl_msg.c:2357 <= fetch input

I (10917) mbedtls: ssl_msg.c:2172 => fetch input

I (10927) mbedtls: ssl_msg.c:2312 in_left: 5, nb_want: 4172

I (10937) mbedtls: ssl_msg.c:2332 in_left: 5, nb_want: 4172

I (10937) mbedtls: ssl_msg.c:2335 ssl->f_recv(_timeout)() returned 4167 (-0xffffefb9)

I (10947) mbedtls: ssl_msg.c:2357 <= fetch input

I (10987) mbedtls: ssl_msg.c:1526 => decrypt buf

I (10987) mbedtls: ssl_msg.c:2138 <= decrypt buf

I (11027) mbedtls: ssl_msg.c:4206 <= read record

I (11027) mbedtls: ssl_tls13_client.c:2249 <= parse encrypted extensions

I (11027) mbedtls: ssl_msg.c:2370 => flush output

I (11037) mbedtls: ssl_msg.c:2379 <= flush output

I (11037) mbedtls: ssl_tls.c:3850 client state: MBEDTLS_SSL_CERTIFICATE_REQUEST

I (11047) mbedtls: ssl_tls13_client.c:2462 => parse certificate request

I (11057) mbedtls: ssl_msg.c:4134 => read record

I (11097) mbedtls: ssl_msg.c:4206 <= read record

I (11097) mbedtls: ssl_tls13_client.c:2492 <= parse certificate request

I (11097) mbedtls: ssl_msg.c:2370 => flush output

I (11107) mbedtls: ssl_msg.c:2379 <= flush output

I (11107) mbedtls: ssl_tls.c:3850 client state: MBEDTLS_SSL_SERVER_CERTIFICATE

I (11117) mbedtls: ssl_tls13_generic.c:820 => parse certificate

I (11127) mbedtls: ssl_msg.c:4134 => read record

I (11127) mbedtls: ssl_msg.c:4202 reuse previously read message

I (11137) mbedtls: ssl_msg.c:4206 <= read record

I (11277) mbedtls: ssl_tls13_generic.c:844 <= parse certificate

I (11277) mbedtls: ssl_msg.c:2370 => flush output

I (11277) mbedtls: ssl_msg.c:2379 <= flush output

I (11277) mbedtls: ssl_tls.c:3850 client state: MBEDTLS_SSL_CERTIFICATE_VERIFY

I (11287) mbedtls: ssl_tls13_generic.c:360 => parse certificate verify

I (11297) mbedtls: ssl_msg.c:4134 => read record

I (11307) mbedtls: ssl_msg.c:4206 <= read record

I (11597) mbedtls: ssl_tls13_generic.c:404 <= parse certificate verify

W (11597) mbedtls: ssl_tls13_generic.c:405 mbedtls_ssl_tls13_process_certificate_verify() returned 0 (-0x0000)

I (11607) mbedtls: ssl_msg.c:2370 => flush output

I (11607) mbedtls: ssl_msg.c:2379 <= flush output

I (11617) mbedtls: ssl_tls.c:3850 client state: MBEDTLS_SSL_SERVER_FINISHED

I (11617) mbedtls: ssl_tls13_generic.c:1244 => parse finished message

I (11627) mbedtls: ssl_msg.c:4134 => read record

I (11637) mbedtls: ssl_msg.c:4206 <= read record

I (11637) mbedtls: ssl_tls13_keys.c:806 => mbedtls_ssl_tls13_calculate_verify_data

I (11657) mbedtls: ssl_tls13_keys.c:837 <= mbedtls_ssl_tls13_calculate_verify_data

I (11657) mbedtls: ssl_tls13_generic.c:1260 <= parse finished message

I (11667) mbedtls: ssl_tls13_keys.c:1609 => derive application traffic keys

I (11677) mbedtls: ssl_tls13_keys.c:1698 <= derive application traffic keys

I (11687) mbedtls: ssl_msg.c:2370 => flush output

I (11687) mbedtls: ssl_msg.c:2379 <= flush output

I (11687) mbedtls: ssl_tls.c:3850 client state: MBEDTLS_SSL_CLIENT_CCS_AFTER_SERVER_FINISHED

I (11697) mbedtls: ssl_tls13_generic.c:1392 => write change cipher spec

I (11707) mbedtls: ssl_msg.c:2960 => write record

I (11717) mbedtls: ssl_msg.c:3097 <= write record

I (11717) mbedtls: ssl_tls13_generic.c:1407 <= write change cipher spec

I (11727) mbedtls: ssl_msg.c:2370 => flush output

I (11727) mbedtls: ssl_msg.c:2384 message length: 6, out_left: 6

I (11737) mbedtls: ssl_msg.c:2391 ssl->f_send() returned 6 (-0xfffffffa)

I (11747) mbedtls: ssl_msg.c:2418 <= flush output

I (11747) mbedtls: ssl_tls.c:3850 client state: MBEDTLS_SSL_CLIENT_CERTIFICATE

W (11757) mbedtls: ssl_tls13_client.c:2579 Switch to handshake traffic keys for outbound traffic

I (11767) mbedtls: ssl_tls13_client.c:2594 skip write certificate

I (11777) mbedtls: ssl_tls13_client.c:2602 skip write certificate verify

I (11787) mbedtls: ssl_msg.c:2370 => flush output

I (11787) mbedtls: ssl_msg.c:2379 <= flush output

I (11797) mbedtls: ssl_tls.c:3850 client state: MBEDTLS_SSL_CLIENT_FINISHED

I (11807) mbedtls: ssl_tls13_generic.c:1322 => write finished message

I (11807) mbedtls: ssl_tls13_keys.c:806 => mbedtls_ssl_tls13_calculate_verify_data

I (11827) mbedtls: ssl_tls13_keys.c:837 <= mbedtls_ssl_tls13_calculate_verify_data

I (11827) mbedtls: ssl_msg.c:2800 => write handshake message

I (11837) mbedtls: ssl_msg.c:2960 => write record

I (11837) mbedtls: ssl_msg.c:949 => encrypt buf

I (11847) mbedtls: ssl_msg.c:1490 <= encrypt buf

I (11847) mbedtls: ssl_msg.c:3097 <= write record

I (11857) mbedtls: ssl_msg.c:2921 <= write handshake message

I (11857) mbedtls: ssl_tls13_generic.c:1339 <= write finished message

I (11867) mbedtls: ssl_tls13_keys.c:1767 => mbedtls_ssl_tls13_compute_resumption_master_secret

I (11877) mbedtls: ssl_tls13_keys.c:1797 <= mbedtls_ssl_tls13_compute_resumption_master_secret

I (11887) mbedtls: ssl_msg.c:2370 => flush output

I (11897) mbedtls: ssl_msg.c:2384 message length: 85, out_left: 85

I (11897) mbedtls: ssl_msg.c:2391 ssl->f_send() returned 85 (-0xffffffab)

I (11907) mbedtls: ssl_msg.c:2418 <= flush output

I (11917) mbedtls: ssl_tls.c:3850 client state: MBEDTLS_SSL_FLUSH_BUFFERS

I (11917) mbedtls: ssl_tls13_client.c:2656 handshake: done

I (11927) mbedtls: ssl_msg.c:2370 => flush output

I (11937) mbedtls: ssl_msg.c:2379 <= flush output

I (11937) mbedtls: ssl_tls.c:3850 client state: MBEDTLS_SSL_HANDSHAKE_WRAPUP

W (11947) mbedtls: ssl_tls13_generic.c:1348 Switch to application keys for inbound traffic

W (11957) mbedtls: ssl_tls13_generic.c:1351 Switch to application keys for outbound traffic

I (11967) mbedtls: ssl_tls.c:3942 <= handshake

I (11967) smtp_example: Verifying peer X.509 certificate...
I (11977) smtp_example: Certificate verified.
I (11977) smtp_example: Cipher suite is TLS1-3-AES-256-GCM-SHA384
I (11987) smtp_example: Authentication...
I (11997) smtp_example: Write AUTH LOGIN
I (11997) mbedtls: ssl_msg.c:5948 => write

I (12007) mbedtls: ssl_msg.c:2960 => write record

I (12007) mbedtls: ssl_msg.c:949 => encrypt buf

I (12017) mbedtls: ssl_msg.c:1490 <= encrypt buf

I (12017) mbedtls: ssl_msg.c:2370 => flush output

I (12027) mbedtls: ssl_msg.c:2384 message length: 37, out_left: 37

I (12037) mbedtls: ssl_msg.c:2391 ssl->f_send() returned 37 (-0xffffffdb)

I (12037) mbedtls: ssl_msg.c:2418 <= flush output

I (12047) mbedtls: ssl_msg.c:3097 <= write record

I (12047) mbedtls: ssl_msg.c:5970 <= write

I (12057) mbedtls: ssl_msg.c:5688 => read

I (12057) mbedtls: ssl_msg.c:4134 => read record

I (12067) mbedtls: ssl_msg.c:2172 => fetch input

I (12067) mbedtls: ssl_msg.c:2312 in_left: 0, nb_want: 5

I (12457) mbedtls: ssl_msg.c:2332 in_left: 0, nb_want: 5

I (12457) mbedtls: ssl_msg.c:2335 ssl->f_recv(_timeout)() returned 5 (-0xfffffffb)

I (12457) mbedtls: ssl_msg.c:2357 <= fetch input

I (12467) mbedtls: ssl_msg.c:2172 => fetch input

I (12467) mbedtls: ssl_msg.c:2312 in_left: 5, nb_want: 544

I (12477) mbedtls: ssl_msg.c:2332 in_left: 5, nb_want: 544

I (12477) mbedtls: ssl_msg.c:2335 ssl->f_recv(_timeout)() returned 539 (-0xfffffde5)

I (12487) mbedtls: ssl_msg.c:2357 <= fetch input

I (12497) mbedtls: ssl_msg.c:1526 => decrypt buf

I (12507) mbedtls: ssl_msg.c:2138 <= decrypt buf

I (12507) mbedtls: ssl_msg.c:4206 <= read record

I (12507) mbedtls: ssl_msg.c:5688 => read

I (12517) mbedtls: ssl_tls.c:3931 => handshake

I (12517) mbedtls: ssl_msg.c:2370 => flush output

I (12527) mbedtls: ssl_msg.c:2379 <= flush output

I (12537) mbedtls: ssl_tls.c:3850 client state: MBEDTLS_SSL_TLS1_3_NEW_SESSION_TICKET

I (12537) mbedtls: ssl_tls13_client.c:2921 => parse new session ticket

I (12547) mbedtls: ssl_msg.c:4134 => read record

I (12557) mbedtls: ssl_msg.c:4202 reuse previously read message

I (12557) mbedtls: ssl_msg.c:4206 <= read record

I (12577) mbedtls: ssl_tls13_client.c:2938 <= parse new session ticket

I (12577) mbedtls: ssl_tls.c:3942 <= handshake

W (12577) mbedtls: ssl_msg.c:5730 mbedtls_ssl_handshake() returned -31488 (-0x7b00)

E (12587) smtp_example: mbedtls_ssl_read failed with error -0x7b00
I (12597) mbedtls: ssl_tls.c:4841 => free

I (12597) mbedtls: ssl_tls.c:4903 <= free

E (12607) smtp_example: Last error was: -0x7b00 - SSL - * Received NewSessionTicket Post Handshake Message. This error code is experimental and may b

More Information.

I also tried to run the example in SSL/TLS mode and got the same result before authentication.

mahavirj commented 3 months ago

@zhichunlee

Have you made any changes of your own in this example? If yes, can you please share the patch here?

Regarding the error you are facing, please try following change once:

diff --git examples/protocols/smtp_client/main/smtp_client_example_main.c examples/protocols/smtp_client/main/smtp_client_example_main.c
index f271c03db0..e02cc12d4d 100644
--- examples/protocols/smtp_client/main/smtp_client_example_main.c
+++ examples/protocols/smtp_client/main/smtp_client_example_main.c
@@ -28,6 +28,7 @@
 #include "mbedtls/error.h"
 #include <mbedtls/base64.h>
 #include <sys/param.h>
+#include "sdkconfig.h"

 /* Constants that are configurable in menuconfig */
@@ -144,6 +145,18 @@ static int write_ssl_and_get_response(mbedtls_ssl_context *ssl, unsigned char *b
         len = DATA_SIZE - 1;
         memset(data, 0, DATA_SIZE);
         ret = mbedtls_ssl_read(ssl, data, len);
+#if CONFIG_MBEDTLS_CLIENT_SSL_SESSION_TICKETS
+        // If a post-handshake message is received, connection state is changed to `MBEDTLS_SSL     _TLS1_3_NEW_SESSION_TICKET`
+        // Call mbedtls_ssl_read() till state is `MBEDTLS_SSL_TLS1_3_NEW_SESSION_TICKET` or ret     urn code is `MBEDTLS_ERR_SSL_RECEIVED_NEW_SESSION_TICKET`
+        // to process session tickets in TLS 1.3 connection
+        if (mbedtls_ssl_get_version_number(ssl) == MBEDTLS_SSL_VERSION_TLS1_3) {
+            while (ret == MBEDTLS_ERR_SSL_RECEIVED_NEW_SESSION_TICKET || ssl->MBEDTLS_PRIVATE(state) == MBEDTLS_SSL_TLS1_3_NEW_SESSION_TICKET) {
+                ESP_LOGD(TAG, "got session ticket in TLS 1.3 connection, retry read");
+                ret = mbedtls_ssl_read(ssl, data, len);
+            }
+        }
+#endif // CONFIG_MBEDTLS_CLIENT_SSL_SESSION_TICKETS
+

         if (ret == MBEDTLS_ERR_SSL_WANT_READ || ret == MBEDTLS_ERR_SSL_WANT_WRITE) {
             continue;

zhichunlee commented 3 months ago

Hi mahavirj Thank you for your confirmation. Based on previous issue#13410, I added psa_crypto_init() before connecting to the mail server. After adding the above processing, the problem has been resolved and the email server has been successfully connected. But a new error occurred while sending the email Content as follows. I (5801) smtp_example: Connected. I (5801) smtp_example: Performing the SSL/TLS handshake... I (6601) smtp_example: Verifying peer X.509 certificate... I (6601) smtp_example: Certificate verified. I (6601) smtp_example: Cipher suite is TLS1-3-AES-256-GCM-SHA384 I (6611) smtp_example: handshake OK, get respone... I (7041) smtp_example: Writing EHLO to server... I (7101) smtp_example: Authentication... I (7101) smtp_example: Write AUTH LOGIN I (7161) smtp_example: Write USER NAME I (7231) smtp_example: Write PASSWORD I (7441) smtp_example: Write MAIL FROM I (7581) smtp_example: Write RCPT I (7681) smtp_example: Write DATA I (7951) smtp_example: Write Content E (8231) smtp_example: Last error was: -0xfffffdd6 - UNKNOWN ERROR CODE (0200) : BASE64 - Output buffer too small

Why did TLS1.3 occur when TLS1.2 did not have this error? Increasing base64_buffer size did not improve either.

Thinks.

espressif / esp-idf

SMTP Client with TLS 1.3 mbedtls_ssl_read failed with error -0x7b00 (IDFGH-12483) #13494

Answers checklist.

IDF version.

Espressif SoC revision.

Operating System used.

How did you build your project?

If you are using Windows, please specify command line type.

Development Kit.

Power Supply used.

What is the expected behavior?

What is the actual behavior?

Steps to reproduce.

CONFIG_MBEDTLS_SSL_PROTO_GMTSSL1_1 is not set

CONFIG_MBEDTLS_SSL_PROTO_DTLS is not set

Debug Logs.

More Information.