SSL support

[anteph]

Hi! I would like to know if you plan to include SSL support in the libraries. I've tryed to send some https requests with no sucess.

It would be cool if it the ESP could act as a secure server too.

I've checked the some examples from the sdk and they have a code to create a secure server with digital certificate.

Thanks!

[igrr]

It occurred to me a few hours ago that my WiFiClientSecure rx path is rubbish. I was able to modify it to support full 16k fragment size, with some minor tweaks to axTLS. Will clean up and do a few more tests, and push tonight.

[sticilface]

that would be phenomenal.... :)

[igrr]

Here's an example which does OTA update over HTTPS from github: https://gist.github.com/igrr/24dd2138e9c8a7daa1b4 Needs latest git version.

[igrr]

@N0TB0T I was able to talk to parse.com using a library by @jcard0na. See my comment in https://github.com/esp8266/Arduino/issues/1029.

[geovanisouza92]

Does it support client certificate yet? I would like to integrate with AWS IoT, that requires it.

[sticilface]

@igrr your example sketch works.
However, I can't get it to work within my sketch. I suspect memory...

===================== START ===================
File Created
HOST: raw.githubusercontent.com:443
State:.sending Client Hello (1)
State:.receiving Server Hello (2)
State:.receiving Certificate (11)
State:.receiving Server Hello Done (14)
State:.sending Client Key Exchange (16)
State:.sending Finished (16)
State:.receiving Finished (16)
E:M 17424

Soft WDT reset

ctx: cont 
sp: 3fff25c0 end: 3fff2fc0 offset: 01b0

[igrr]

E:M 17424 This is a memory issue indeed. I'll try to reduce memory footprint further.

[sticilface]

I'm running with quite a bit of heap spare, how much is actually required?

Free Heap: 26872
Sat Nov 21 22:40:26 2015

[igrr]

I think the issue is not the total amount of free memory available, it's the size of continuous available space. Memory gets fragmented, so while you do have 26k free, you may not have 17k available as a single chunk. Actually this 17k get allocated after handshake, which complicates things even more. Will look into this.

[sticilface]

cool beans:) much appreciated. all work in progress and thanks for the hard work!

[ekarlso]

Hey, is there any update on this ? I wanna test out TLS with MQTT for my sketch. Awesome work btw :)

[GjBob]

Anyone has succeded in integrating with aws iot? So far i was able to do https request but as i understood the current implementation is using tls 1.1 and aws iot require tls 1.2

[Apple456]

would you mind to share some code for https intergation? I still didn't manage to perform post on https due some memory issues..

[Links2004]

HTTPClient http;
http.begin("https://domain/post.php");
http.addHeader("Content-Type", "application/x-www-form-urlencoded");
http.POST("from=test");
http.writeToStream(&Serial);
http.end();

[svdgraaf]

@GjBob yeah, that sucks :( I see that the REST endpoint does support tls 1.1 (and even 1.0):

$ openssl s_client -connect [device-id].iot.eu-west-1.amazonaws.com:443 -tls1
CONNECTED(00000003)

What would we need to get tls1.2 running? Is there something we can help with? Or is it just not feasible at all with the given hardware?

[ekarlso]

Anyone of you wanna take the challenge of getting the latest code working with TLS1.1 and MQTT ?

[igrr]

@svdgraaf if you could add TLS 1.2 support to axTLS library, that would be great.

[abl]

https://www.wolfssl.com/wolfSSL/Blog/Entries/2010/12/14_A_Comparison_of_TLS_1.1_and_TLS_1.2.html

At a glance, doable, but not trivial. It might be better to move away from axTLS and on to a different TLS implementation entirely. :(

[geovanisouza92]

aws/aws-iot-device-sdk-embedded-C#7 They is taking a look at this.

[svdgraaf]

I think it should be doable as a community project, I'm nowhere near smart enough, and wouldn't know where to start :) I think switching to a different tls implementation is the fastest and easiest way to go. We could pull this out of this package, and create a seperate implementation, to circumvent any potential licensing issues...

I will create a PR to at least implement sha265(hmac) (there are multiple implementations available), but that's just a small part of the whole implementation.

[jcard0na]

Hi,

I just thought I'd mention that these guys are using mbed TLS on the esp8266 which seems to support TLS 1.2.

Their TLS example connects to an HTTP server over TLS 1.2 and worked fine for me.

This TLS library was discussed earlier in this thread by @igrr and @abl and was dismissed due to licensing concerns.

It appears that mbed TLS has been re-licensed to the Apache license since that exchange took place. Maybe it is time to reconsider switching from axTLS?

Cheers,

[igrr]

@projectgus has mentioned an issue with mbed TLS — it has separate RX and TX buffers, and due to a design decisions found in this library, both must be 16k (not just the RX one). Giving 32k RAM to TLS library is just not feasible on the ESP8266. So in order to use mbed TLS on the ESP8266 one must reduce maximum supported fragment size. We know that this breaks some applications (search this thread for "fragment size"). For instance, OTA over HTTPS only works when we have full 16k fragment size support. This is because most TLS servers out there do not support fragment size negotiation extension.

[svdgraaf]

Yep, they do not support OTA atm it seems: https://github.com/SuperHouse/esp-open-rtos/issues/20 and https://github.com/SuperHouse/esp-open-rtos/issues/10

Could be a tradeoff choice though, I don't really care for OTA for the project I'm doing, but love tls1.2, so as I said, if we can build something which is pluggable, someone could "just" switch the WifiClientSecure for another implementation :)

[projectgus]

Hello! One idea I have had regarding the buffer size problem is to add "asymmetric buffer" support to mbedTLS, so it can have a 16K RX buffer and a much smaller (minimum size is 512 bytes from memory) TX buffer. I've had a glance around in the code and it looks do-able at a first glance. I haven't had time to try and implement it though.

Regarding OTA support, esp-open-rtos can do OTA over plaintext TFTP. I hit problems doing OTA over TCP (ie including TLS) but they are related to WiFi frames being dropped while writing to flash, rather than problems with TLS support itself. So a different problem, albeit still a problem. Seems like this is sorted when using newer Espressif SDKs, or maybe something clever that @igrr & team have done!

[ekarlso]

Curious though, have anyone looked at using the route that Particle / Digistump Oak (esp8266 esp-12 alike module) are doing for OTA using CoaP over DTLS?

2015-12-10 12:08 GMT+01:00 Angus Gratton notifications@github.com:

Hello! One idea I have had regarding the buffer size problem is to add "asymmetric buffer" support to mbedTLS, so it can have a 16K RX buffer and a much smaller (minimum size is 512 bytes from memory) TX buffer. I've had a glance around in the code and it looks do-able at a first glance. I haven't had time to try and implement it though.

Regarding OTA support, esp-open-rtos can do OTA over plaintext TFTP. I hit problems doing OTA over TCP (ie including TLS) but they are related to WiFi frames being dropped while writing to flash, rather than problems with TLS support itself. So a different problem, albeit still a problem. Seems like this is sorted when using newer Espressif SDKs, or maybe something clever that @igrr https://github.com/igrr & team have done!

— Reply to this email directly or view it on GitHub https://github.com/esp8266/Arduino/issues/43#issuecomment-163580436.

[madsci1016]

Just tested WiFiClientSecure() and it worked great, thanks. My application is a IoT sensing application, and I'd like to enable 'client authentication' so the server verifies it's my ESP8266 trying to publish new sensor data.

Some googling shows axTLS supports client authentication, but before i lock myself in a room for a week to go through the source code, I'd figure I'd ask. Does the current port of WiFiClientSecure() have a way to support client authentication? IE, I'd have to embed a trusted cert in the firmware that would be presented to the server during the handshake.

[Links2004]

you may search this functions: https://github.com/esp8266/Arduino/blob/master/libraries/ESP8266WiFi/src/WiFiClientSecure.h#L52-L66

[madsci1016]

Thanks Links, that's a start.

[madsci1016]

@igrr one of your future milestones was "release memory allocated for certificate storage after check is complete" has that been implemented yet?

I'm now running into a memory issue:

State:    receiving Certificate (11)
crypto/bigint.c:1072 realloc 1032, left 15480
crypto/bigint.c:1072 realloc 1032, left 11944
crypto/bigint.c:1072 realloc 1028, left 7584
crypto/bigint.c:1072 realloc 2056, left 5424
crypto/bigint.c:1072 realloc 1032, left 4344
crypto/bigint.c:1072 realloc 1024, left 3832
crypto/bigint.c:1072 realloc 1032, left 1048
crypto/bigint.c:1072 realloc 512 failed, left 752
Fatal exception 29(StoreProhibitedCause):

I try to malloc them memory I need after the handshake, but at that point ESP.getFreeHeap() only shows 12KB available. Before the client.conneect call it is 27KB+. I

[igrr]

It is implemented in 2.1.0. After WiFiClient::connect returns, certificate will be kept until the first call to read or write functions. This is done to allow one call verify function and check the certificate. Once you call read or write, certificate will be released.

[apigeoje]

@igrr - You're frikin awesome. I've just spent 15 minutes reading this. I haven't even tried to use it yet but I'm already grateful for all your work...

[madsci1016]

Yes, @igrr is super awesome and needs to show us a paypal donate button so we can show our appreciation.

I've added some of my own debug as well as turned on the builtin. Here's what I got:

`Free Mem before connection: 38536

ssl/tls1.c:549 malloc 6864, left 30960

please start sntp first !

State: sending Client Hello (1)

State: receiving Server Hello (2)

State: receiving Certificate (11)

crypto/bigint.c:1072 realloc 1032, left 26584

crypto/bigint.c:1072 realloc 1032, left 23048

crypto/bigint.c:1072 realloc 1028, left 18688

crypto/bigint.c:1072 realloc 2056, left 16528

crypto/bigint.c:1072 realloc 1032, left 15448

crypto/bigint.c:1072 realloc 1024, left 14936

crypto/bigint.c:1072 realloc 1032, left 12024

State: receiving Server Hello Done (14)

crypto/bigint.c:1072 realloc 1024, left 8432

State: sending Client Key Exchange (16)

State: sending Finished (16)

State: receiving Finished (16)

Free Mem after handshake: 12184

ssl/tls1.c:1422 malloc 17408, left 19280

Free Mem after print: 19648

Response: :rd 5, 330, 0

Client closing connection

Alert: close notify`

So with 38KB free originally, 30KB are consumed during the handshake and 4KB are given back right away, leaving the user with 12KB free. After the first write call, an additional 7KB are given back, bringing free mem to 19KB with the connection open.

@igrr Is this the expected behavior? Are there any other improvements you could suggest on the server end to optimize memory usage?

My application is a IoT sensor that will include a 1 or 2 second WAV file recorded by the ESP8266, That's why I'm being a stickler for memory.

[krebbi]

is there a known issue with Let's Encrypt certificates?

because I get

please start sntp first ! State: sending Client Hello (1) Alert: handshake failure Error: SSL error 40 Alert: unexpected message Alert: close notify

[igrr]

@krebbi does the server allow TLS 1.1? axTLS doesn't support TLS 1.2, which may be the reason for this error.

[krebbi]

TLS 1.0, 1.1 and 1.2 https://api.12view.me

[krebbi]

or is RSA 4096 bits too big?

[madsci1016]

I believe this ESP version only supports AES 128 or 256

[krebbi]

these Cipher Suites are supported by my cert:

TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030) ECDH secp256r1 (eq. 3072 bits RSA) FS 256 TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xc02f) ECDH secp256r1 (eq. 3072 bits RSA) FS 128 TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (0x9f) DH 2048 bits FS 256 TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (0x9e) DH 2048 bits FS 128 TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (0xc028) ECDH secp256r1 (eq. 3072 bits RSA) FS 256 TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA (0xc014) ECDH secp256r1 (eq. 3072 bits RSA) FS 256 TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 (0x6b) DH 2048 bits FS 256 TLS_DHE_RSA_WITH_AES_256_CBC_SHA (0x39) DH 2048 bits FS 256

[slaff]

@igrr Do you know if NGINX supports max_fragment_length negotiation ?

[electronicsguy]

@igrr Could someone please tell me if https can work on the ESP-01?

[svdgraaf]

@electronicsguy some https connections will work, some won't. It has to do with the TLS version, the used ciphers, and/or the configuration of the webserver.

[slaff]

..." Nginx hardcodes 16KB size in ngx_event_openssl, which you can change and recompile from source."

Do you know if NGINX supports max_fragment_length negotiation ?

It turns out that you can set SSL fragment size in NGINX (>= 1.5.9 ). The directive is called ssl_buffer_size. And there is no need to recompile NGINX.

[electronicsguy]

@svdgraaf @igrr Thanks for the follow-up. I am still trying to understand the details. All I want to do is send data from esp8266 directly into a google spreadsheet (without using a http to https converter service like pushingbox). So is incorporating support for TLS1.2 unachieveable with the current esp hardware? Or is it a matter of someone writing up that code, which has just not been done as yet?

[AdamMiltonBarker]

Anyone got TLS1.2 working yet and a way to add trusted certs ?

[fsommer1968]

@AdamMiltonBarker! Have a look at WolfSSL, they support TLS V1.2 but I´m not sure whether the library works with ESP. I want to use TLS 1.2 with Bluemix IoT but I have currently not the time to test. Maybe you can try?

[marvinroger]

wolfSSL has already been discussed, and the problem is it is GPL and not LGPL. I am not sure I understand the difference, to be honest.

[fsommer1968]

For my project it would´nt be important whether GPL or LGPL.

[AdamMiltonBarker]

I agree I think security is more important that the license.

[DaKaZ]

@igrr - I too greatly appreciate all you do for the esp8266 community. Having been developing on this hardware for the last year I could not have done it without you!

I am having trouble with the WiFiClientSecure implementation though. I basically have the exact same code for HTTP as HTTPS but I never get the response back from the server with SSL (unless I remove connection: close from the http headers and wait for a timeout). I have verified that the server is indeed sending a response back (wireshark). The code is here: https://gist.github.com/DaKaZ/5044f3168412ce0b3f5bf1d46130765e

From my debug output, I see the Alert: close notify before I even call sslClient.connected() or sslClient.available() so I fear I am missing something. Any ideas?? Here is a little more from my client debug:

HTTP: connect
please start sntp first !
State:  sending Client Hello (1)
State:  receiving Server Hello (2)
State:  receiving Certificate (11)
State:  receiving Server Hello Done (14)
State:  sending Client Key Exchange (16)
State:  sending Finished (16)
State:  receiving Finished (16)
Verifiying SSL certificate
SSL certificate matches
HTTP: connected
REQUEST: 

SSL Print: POST
SSL Print:  
SSL Print: /api/v1/plugs
SSL Print:  HTTP/1.1

SSL Print: Accept-Encoding: application/json
SSL Print: 

SSL Print: Timestamp: 1462488709
SSL Print: 

SSL Print: Signature: <removed for online post>
SSL Print: 

SSL Print: Host: 
SSL Print: <removed for online post>
SSL Print: 

SSL Print: Connection: close

SSL Print: Content-Length: 31

SSL Print: Content-Type: 
SSL Print: application/json
SSL Print: 

SSL Print: 

SSL Print: { "mac" : "<removed for online post>" }
SSL Print: 

SSL Print: 
Alert: close notify

END REQUEST
HTTP: call readResponse
HTTP: NON-NULL RESPONSE POINTER: 
HTTP: RESPONSE: 
HTTP: Connect: 0 Available: 0
HTTPS client closed 
HTTP: return readResponse3
HTTP: return readResponse
HTTP: stop client
HTTP: client stopped
Got response: 
pm open,type:2 0

[chaeplin]

@DaKaZ I get body of response using @igrr's example https://github.com/esp8266/Arduino/blob/master/libraries/ESP8266WiFi/examples/HTTPSRequest/HTTPSRequest.ino#L61-L84