espressif / esp-idf

Espressif IoT Development Framework. Official development framework for Espressif SoCs.
Apache License 2.0
13.35k stars 7.21k forks source link

Differences between using lwip and mbedTLS vs. native sockets and mbedTLS on Linux (Ubuntu 19.04) (IDFGH-1221) #3523

Closed PerMalmberg closed 5 years ago

PerMalmberg commented 5 years ago

Hi,

Some background for those that don't know: I develop a C++ framework for the ESP32 on top of ESP-IDF, with the main purpose of being event driven, nearly memory static and allowing applications being compiled/development natively on Linux (to allow the use of Adress Sanitizers etc) before being compiled using xtensa-gcc8 and then run on the ESP32.

I've just implemented server-side socket management which I've used to build an HTTP-server. I also added support for TLS (using mbedTLS).

Now, to my problem:

Things are working just fine both with and without TLS as long things run natively on Linux. Non-TLS works fine on the ESP32, but anything using TLS on the ESP32 fails.

Compiling the same code and running it on the the ESP32 the result is this:

> wget --verbose --no-check-certificate https://192.168.10.94:8443
--2019-05-22 20:07:51--  https://192.168.10.94:8443/
Connecting to 192.168.10.94:8443... connected.
Unable to establish SSL connection.
> curl --cacert root_ca.crt  -vv https://192.168.10.94:8443
* Expire in 0 ms for 6 (transfer 0x55eae4b084f0)
*   Trying 192.168.10.94...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x55eae4b084f0)
* Connected to 192.168.10.94 (192.168.10.94) port 8443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: root_ca.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 192.168.10.94:8443 
* Closing connection 0
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 192.168.10.94:8443

What baffles me is that OpenSSL is able to do a handshake from the same machine curl and wget are run: openssl s_client -connect 192.168.10.94:8443 -CAfile root_ca.crt

Can't use SSL_get_servername
depth=1 C = SE, L = Asteroid 51, O = Secret Universe Inc., CN = localhost
verify return:1
depth=0 C = SE, CN = localhost
verify return:1
write:errno=0
---
Certificate chain
 0 s:C = SE, CN = localhost
   i:C = SE, L = Asteroid 51, O = Secret Universe Inc., CN = localhost
---
Server certificate
-----BEGIN CERTIFICATE-----
MIID/jCCAeYCFDRAbCBUb0xdLkG/sSTD+hzNXG71MA0GCSqGSIb3DQEBCwUAMFYx
CzAJBgNVBAYTAlNFMRQwEgYDVQQHDAtBc3Rlcm9pZCA1MTEdMBsGA1UECgwUU2Vj
cmV0IFVuaXZlcnNlIEluYy4xEjAQBgNVBAMMCWxvY2FsaG9zdDAeFw0xOTA0MTYx
ODEzMTZaFw0yOTA0MTMxODEzMTZaMCExCzAJBgNVBAYTAlNFMRIwEAYDVQQDDAls
b2NhbGhvc3QwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDAjN54YcmZ
OwW7WahNjrlg9TQQqP35BDwKj56jMQ93IP5iF6xoYksi93kP4Eg4sE9jzLo/weOA
19STM9DSA6BrZhmLTViwH3T2J3/iZU1Q2QMFDeZwFvRXesYIdfdJLTWqCXy5wD9E
8MuU8HW9gk35/ChDfXH2ylaVl3dL+5rTEBEA0b2aOA0qhrfpvdzEFh8kHVw27Uwe
Huh7a2ko7fWwpWXMG/KusPBdVEhhPS5NolnnLDnoIj3z9MSJAi5cNdxJaJHVqvfA
t2Y1Viy+CP2oERswJAcAoQ8Mn712dr2EYpcIfqljSM9mgj9x1oAKqQrJHD+i6vGS
ilTpkCMdOOd/AgMBAAEwDQYJKoZIhvcNAQELBQADggIBAEUYLx0jSQT79+wdvwPh
O9xDyEIUc2Ih4XV19FUw3TrhBV92q+1M81lv60lTOl5fepvFG7XInav5hHs61Wx/
IoNYWpCinapbU3E+2R6FOcTP1iTzfbeZwjcaIfXQ65ilX6j0T0oYAsfg8Y63kYKe
miqPkkFNY0upcnyir/Z220urYm9+mT4HOPT4mNqMgXmLMlPffK8c1Fr2NXQMvsVz
nEhxGYVQmqCbxceVkJD6swATtspc66GirBkAjuPfJOvCDfAHkzA3BZ26xziQdY8T
UWCBtu4r5mhiWpU46L8f1CtG4RmRoi8g3STA7DaMM44Kl30lyBGT0vRfLyPU6vb3
TCI4tdx5PFb46p1jeLL0IRShZ9pn/MPAja1+GE4k5/6WgQ2vVxf/90lBc7CbMrQ9
CBMffs2HQKBwIYaPDtqoh9WHYZDv5TcVbfFrwtHNxRHWWofCxilSpjfjsoDwWdT/
Imd+LMPdgUmA+dErjErrQ2uXLlrY2TVplxBaXUbeO+qMyrCNZIe1Ej5A99wyi2vW
ydLYfWh34mAYiJEidqedt0TbbdwEAB8uO2XS0LSq4FBfPmXxhKgCSc99FxScVGi/
xW1bZVeeORuAPAzGV2lMfGqIst59QXq7G5O4v2SgRyJ3tm+2scWyzfNwRqa43r9s
s/WzJmP8eZyppmRvbTiYdBcy
-----END CERTIFICATE-----
subject=C = SE, CN = localhost

issuer=C = SE, L = Asteroid 51, O = Secret Universe Inc., CN = localhost

---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA
Server Temp Key: ECDH, P-521, 521 bits
---
SSL handshake has read 1543 bytes and written 293 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : 0000
    Session-ID: 51631DB201385B19975B95DF29DC9C672AD59855435680C2FCA0EF6A3BBC884F
    Session-ID-ctx: 
    Master-Key: 
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1558547434
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: yes

Question: Am I being naive thinking that if my usage of mbedTLS works on native Linux, it should also work on the ESP32? If yes, what are the differences between how sockets and mbedTLS works on the ESP32 vs native Linux that I've missed? I have read the docs for both IDF and mbedTLS more times than I care to count, but I've seen no mention of such differences.

The most relevant code is SecureSocket, but I don't really expect anyone to read through it all since it requires the entire framework to build and run.

For the brave and curious, here's a short summary of whats needed to get it to run:

Clone the https://github.com/PerMalmberg/Smooth/tree/feature/15-ServerSocket branch, and then build the http_server_test (pre-selected in smooth/test/CMakeList.txt) by running these commands in <location of repo>/build. Be sure to use the IDF-version and gcc mentioned at the beginning of this post.

make -j8
~/esp/esp-idf/tools/idf.py -C .. -p /dev/ttyUSB1 app-flash monitor

You'll have to prepare an SDCard with the web_root folder, for example the one found in smooth/test/http_server_test/static_content or just create your own index.html page. I'm using a WROOVER DevKit 3.x board so if your SD Card uses different pins on the ESP32 you can changes those on line 148 of smooth/test/http_server_test/http_server_test.cpp

Also, change the SSID and password in smooth/test/http_server_test/wifi_creds.h.

To build it for native Linux, just compile it from the same location as any other CMake project.

jitin17 commented 5 years ago

@PerMalmberg Since openssl s_client yields expected results, I speculate that the problem is not with something related to ESP32 but with a self-signed certificate.

To see if this is true, can you supply -k(allows insecure connections) option to the curl command, something like this curl -k --cacert root_ca.crt -vv <url>

PerMalmberg commented 5 years ago

@jitin17 Without -k, curl is expected to reject the certificate, as per above output it doesn't get far enough in the handshake to do that. But, just for the record, here is the output you asked for:

> curl -vv -k https://192.168.10.94:8443      
* Expire in 0 ms for 6 (transfer 0x5558316384f0)
*   Trying 192.168.10.94...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x5558316384f0)
* Connected to 192.168.10.94 (192.168.10.94) port 8443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 192.168.10.94:8443 
* Closing connection 0
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 192.168.10.94:8443
negativekelvin commented 5 years ago

What is the mbedtls debug output from esp32?

PerMalmberg commented 5 years ago

What is the mbedtls debug output from esp32?

Right, I thought I wrote a note about that in the original post. I've been unable to enable mbedTLS debugging on the ESP, even after enabling it via menuconfig. I can't get it to link (undefined references to the debug functions). I've tried hard coding it to be enabled in the mbedtls component, but I can't get it to output anything.

PerMalmberg commented 5 years ago

I can now answer my own question: No, I'm not naive to think the same implementation of mbedTLS should works on both Linux and ESP32.

The problem has turned out to be that the TLS handshake takes nearly 5 seconds and I had a timeout of 1.5s and didn't log when the socket was closed.