search

esp8266 / Arduino

ESP8266 core for Arduino
GNU Lesser General Public License v2.1
16.08k stars 13.33k forks source link

SSL support #43

Closed anteph closed 8 years ago

anteph commented 9 years ago

Hi! I would like to know if you plan to include SSL support in the libraries. I've tryed to send some https requests with no sucess.

It would be cool if it the ESP could act as a secure server too.

I've checked the some examples from the sdk and they have a code to create a secure server with digital certificate.

Thanks!

grahamehorner commented 9 years ago

just completed my first RESTful call to Azure Service Bus over HTTPS; thanks :dancer: +1

N0TB0T commented 9 years ago

I'm sorry, but I still get a "'WiFiClientSecure' was not declared in this scope" - the most current version of Staging (packaged) seems to be 1.6.5-1160-gef26c5f, which I installed over the BoardManager. Doesn't it work when just adding the link in the IDE Settings, do I have to build it on my own? Any help appreciated, thx

N0TB0T commented 9 years ago

And @igrr - Thanks for the effort on SSL anyway, that's exactly what I needed!

timkay commented 9 years ago

You need another include, as I show here. 

#include <ESP8266WiFi.h>
#define USE_SSL 1
#if USE_SSL
#   include <WiFiClientSecure.h>
    WiFiClientSecure client;
#   define PORT 443
#else
    WiFiClient client;
#   define PORT 80
#endif
N0TB0T commented 9 years ago

Found that out that very moment, digging into ESP8266WiFi.c - =)

Still: Thanks a lot!

timkay commented 9 years ago

I can't seem to open two WiFiClientSecure connections at the same time. The second connect() causes a WDT reset. See https://www.dropbox.com/s/i8pin2m34shhxn1/esp_test_two_tls.ino?dl=0

scandone
state: 0 -> 2 (b0)
state: 2 -> 3 (0)
state: 3 -> 5 (10)
add 0
aid 1
cnt 

connected with 688 Berry Ave, channel 11
dhcp client start...
ip:192.168.8.121,mask:255.255.255.0,gw:192.168.8.254
About to connect() to www.yahoo.com port 443 uri /error
connected
About to connect() to www.yahoo.com port 443 uri /error
E:M 1048

Soft WDT reset

ctx: cont 
sp: 3ffeb410 end: 3ffeba20 offset: 01b0

>>>stack>>>
3ffeb5c0:  3fffb8e0 401000a2 3fff9a40 4022c0a7  
3ffeb5d0:  3fff9128 3fff9a60 00000081 4022cd0d
igrr commented 9 years ago

With two connections you are running out of memory, so only one connection at a time for now. I think Espressif's version of axTLS shares this limitation.

On Tue, Oct 6, 2015, 08:23 Timothy Kay notifications@github.com wrote:

I can't seem to open two WiFiClientSecure connections at the same time. The second connect() causes a WDT reset. See https://www.dropbox.com/s/u6qma92jwaa65pr/esp_test.ino?dl=0

scandone state: 0 -> 2 (b0) state: 2 -> 3 (0) state: 3 -> 5 (10) add 0 aid 1 cnt

connected with 688 Berry Ave, channel 11 dhcp client start... ip:192.168.8.121,mask:255.255.255.0,gw:192.168.8.254 About to connect() to www.yahoo.com port 443 uri /error connected About to connect() to www.yahoo.com port 443 uri /error E:M 1048

Soft WDT reset

ctx: cont sp: 3ffeb410 end: 3ffeba20 offset: 01b0

stack>>> 3ffeb5c0: 3fffb8e0 401000a2 3fff9a40 4022c0a7 3ffeb5d0: 3fff9128 3fff9a60 00000081 4022cd0d

— Reply to this email directly or view it on GitHub https://github.com/esp8266/Arduino/issues/43#issuecomment-145744452.

timkay commented 9 years ago

I also seem to have trouble reading a stream with packets larger than about 1500 bytes. I turned on DEBUGV and get error :er -9 53 1, which is ERR_RST, but I can't find the code that makes that call. See https://www.dropbox.com/s/1d43d0li1cv7io5/esp_test_big_packet.ino?dl=0

scandone
state: 0 -> 2 (b0)
state: 2 -> 3 (0)
state: 3 -> 5 (10)
add 0
aid 1
cnt 

connected with 688 Berry Ave, channel 11
wifi evt: 0
dhcp client start...
wifi evt: 3
ip:192.168.8.121,mask:255.255.255.0,gw:192.168.8.254
About to connect() to demonstration.firebaseio.com port 443 uri /bigpacket.json
:ref 1
:wr
:sent 56
:ww
:rn 1460
:rd 5, 1460, 0
:rdi 1460, 5
:rd 74, 1460, 5
:rdi 1455, 74
:rd 5, 1460, 79
:rdi 1381, 5
:rd 1376, 1460, 84
:rdi 1376, 1376
:c0 1376, 1460
:rn 1460
:rd 1460, 1460, 0
:rdi 1460, 1460
:c0 1460, 1460
:rn 362
:rd 353, 362, 0
:rdi 362, 353
:rd 5, 362, 353
:rdi 9, 5
:rd 4, 362, 358
:rdi 4, 4
:c0 4, 362
:wr
:sent 267
:ww
:wr
:sent 6
:ww
:wr
:sent 69
:rn 75
:ww
:rd 5, 75, 0
:rdi 75, 5
:rd 1, 75, 5
:rdi 70, 1
:rd 5, 75, 6
:rdi 69, 5
:rd 64, 75, 11
:rdi 64, 64
:c0 64, 75
connected
:wr
:sent 53
:ww
:wr
:sent 69
:ww
:wr
:sent 53
:ww
:wr
:sent 53
:ww
:wr
:sent 85
:ww
:wr
:sent 53
:ww
:wr
:sent 53
:ww
.:rn 229
:rch 229, 1460
:rch 1689, 1460
:rch 3149, 1460
:rd 5, 4609, 0
:rdi 229, 5
.:rd 224, 4609, 5
:rdi 224, 224
:c 224, 229, 4609
*** rb: 0 + 187 = 187
[128]HTTP/1.1 200 OK
Content-Length: 12842
Strict-Transport-Security: max-age=31556926; includeSubDomains; preload
Content-Type: a:rd 5, 4380, 0
:rdi 1460, 5
:wr
:sent 53
:rch 4380, 794
:ww
:wr
:er -9 53 1
:ww
[59]pplication/json; charset=utf-8
Cache-Control: no-cache
timkay commented 9 years ago

I updated the previous comment with more information.

Using https://www.dropbox.com/s/krjrv69czlglsuh/esp_test_mid_packet.ino?dl=0, I find that it works if the data length is <= 5067, but it gives an error:

WiFiClientSecure rx overflow*** rb: 0 + 1535 = 1535
timkay commented 9 years ago

Thanks! Could we get you to update staging?

igrr commented 9 years ago

That commit doesn't fix the issue you have reported.

ts-mini commented 9 years ago

I'm unsure of what is really happening - but it seems like any Amazon Cloudfront SSL doesn't work with axTLS.

Their supported list of ciphers are: ECDHE-RSA-AES128-SHA ECDHE-RSA-AES256-SHA AES256-SHA AES128-SHA DES-CBC3-SHA RC4-MD5

cottsak commented 9 years ago

@igrr Still getting crashes when making POST over HTTPS sporadically:

making POST request to Twilio for sending sms..

Soft WDT reset

ctx: cont 
sp: 3ffeb800 end: 3ffebe10 offset: 01b0

>>>stack>>>
3ffeb9b0:  4022bea3 00000030 00000010 4022c973  
3ffeb9c0:  bd92dada 3fff3f88 3fff9ed8 4022cb0c  
3ffeb9d0:  40101635 3fffb238 3fffb238 3fff3f88  
3ffeb9e0:  00000080 00000102 3fff9ba0 4022cdbe  
3ffeb9f0:  3fff5288 ffffff81 00000000 00000081  
3ffeba00:  000218f5 0000007f 00000000 00000081  
3ffeba10:  3fffaa10 3fff3f88 3fff3b88 3fff40b0  
3ffeba20:  3fffb64c ffffffff 3fff3f88 3fff3f88  
3ffeba30:  00000080 3fff3f88 3fff9ba0 3fff3f88  
3ffeba40:  00000080 3fff3f88 3fff9ba0 4022d926  
3ffeba50:  3fff4050 0000007f 5aecfcb7 00000001  
3ffeba60:  3fff3f88 3fff9b80 3fff9ba0 00000001  
3ffeba70:  00000010 3fff3f88 0000000f 4022db81  
3ffeba80:  3fff3b88 3fff4090 0000000f 00000001  
3ffeba90:  00000001 3fff40f0 00000000 4022c9d8  
3ffebaa0:  00000200 3fff5c53 3fff5c53 00000000  
3ffebab0:  00000200 3fff5c53 3fff4018 4022ed2c  
3ffebac0:  00000000 000001d0 3ffebaf0 00000030  
3ffebad0:  066ce839 ac113df1 4b66993c 00000004  
3ffebae0:  3fff7020 3fff5c20 3fff5c4d 4022ad82  
3ffebaf0:  ec980203 86ffba2d 8634bd31 842a26f6  
3ffebb00:  6a6dbffb 54a77dc7 5fd8c70d a6b83ed6  
3ffebb10:  86c3f654 3986aa77 05c35ee6 9efce561  
3ffebb20:  3fff5c4d 3fff7020 3fff5c20 4022b115  
3ffebb30:  337c38c1 5a926727 1cf0d2cd e36493c8  
3ffebb40:  e4c18e1c 29db7716 0b99662c 25956359  
3ffebb50:  2ef9f8a0 3fff3c60 3fff3c20 00001140  
3ffebb60:  00000004 3fff5c4d 00000003 3fff3c60  
3ffebb70:  3fff5c20 3fff5c4d 00000004 00000004  
3ffebb80:  3fff5c4d 3fff7020 3fff5c20 4022abaa  
3ffebb90:  00000038 00000000 3fff3ab8 402048fa  
3ffebba0:  00000038 00000001 00000033 3fff5c48  
3ffebbb0:  00000000 00020010 0000000e 00000010  
3ffebbc0:  00000004 401000a2 3ffebd10 40204958  
3ffebbd0:  3fff7020 00000033 00000016 00000000  
3ffebbe0:  3fff7020 3fff5c20 3fff5c20 4022ace6  
3ffebbf0:  3fff7020 3fff5c4d 3fff5c20 4022aebd  
3ffebc00:  00000000 3fff3af0 3fff5c20 4022af46  
3ffebc10:  3fff5600 40205822 3ffebd10 3ffead0c  
3ffebc20:  3fff3b60 3fff3848 3ffebd10 4020475a  
3ffebc30:  384ff336 3ffebd10 402017ec 3ffebd10  
3ffebc40:  3ffebd10 000001bb 3ffebd10 402049cc  
3ffebc50:  3ffe9900 384ff336 3ffec2d0 40104b54  
3ffebc60:  40102e58 000001bb 3ffebd10 40203ca9  
3ffebc70:  3ffe9900 384ff336 3ffe9900 384ff336  
3ffebc80:  3ffe93f2 00000038 00000000 7fffffff  
3ffebc90:  0000007f 3ffe93f2 3ffebe68 402049e4  
3ffebca0:  3ffebd10 40240000 3ffebe68 40202448  
3ffebcb0:  6c4e5551 6a646a59 3545324e 6a56474e  
3ffebcc0:  68686a59 684a6d5a 324d6a5a 7a4d7a59  
3ffebcd0:  3551574f 6b5a544e 7a45444e 326f6a4e  
3ffebce0:  68686a5a 6964544e 31417a4e 78597a4e  
3ffebcf0:  6946574e 32457a59 31416a59 6d5a7a4d  
3ffebd00:  33553259 3045474d 3d41414f 40202400  
3ffebd10:  3ffe9790 00000000 000003e8 00000001  
3ffebd20:  00000000 3fff3ab8 3fff3848 00000040  
3ffebd30:  00000040 00000001 3ffebd90 40207ed2  
3ffebd40:  3fff3a58 00000040 3ffebd90 40207f2a  
3ffebd50:  30312e30 00000000 3ffebd90 40207f5a  
3ffebd60:  3ffebd90 40240000 3ffebd90 4020805c  
3ffebd70:  3ffe9634 40240000 00000000 3ffe8000  
3ffebd80:  3ffeaa98 40240000 00000000 402026e3  
3ffebd90:  3fff3728 0000004e 00000040 3fff3728  
3ffebda0:  0000003f 00000036 3fff3a58 0000004f  
3ffebdb0:  00000040 3ffeaad0 3ffebe68 3ffebe3c  
3ffebdc0:  3fffdc20 3ffeaad0 3ffebe68 3ffebe3c  
3ffebdd0:  3ffea998 00000000 3ffea998 4020290e  
3ffebde0:  3fffdc20 00000000 3ffebe34 402022f8  
3ffebdf0:  00000000 00000000 3ffebe34 4020186a  
3ffebe00:  00000000 00000000 3ffeadf0 40100398  
<<<stack<<<

 ets Jan  8 2013,rst cause:2, boot mode:(3,6)

load 0x4010f000, len 1264, room 16 
tail 0
chksum 0x42
csum 0x42
~ld

Source is https://github.com/cottsak/opensesameseed/blob/master/iforgottocloseit/iforgottocloseit.ino

grahamehorner commented 9 years ago

this may help; just prior to the crash the debug output is :-

making POST request to Twilio for sending sms.. chg_B:-60 E:M 536

Soft WDT reset ...

making POST request to Twilio for sending sms.. State: sending Client Hello (1) State: receiving Server Hello (2) State: receiving Certificate (11) State: receiving Server Hello Done (14) E:M 1048

cottsak commented 9 years ago

@grahamehorner what are those? did you mod my code and dump that?

igrr commented 9 years ago

@timkay Sorry I haven't been able to follow this issue; still on my vacation.

I tried your sketch with a bit more debug info enabled, and it turned out axTLS is unable to process TLS record because the buffer size is not sufficient. TLS record from demonstation.firebaseio.com comes with size=8240, while local buffer is 5115 bytes (this is hard-coded in my axTLS build at the moment, RT_MAX_PLAIN_LENGTH is set to 4096). So axTLS sees that it can't receive the whole record and bails out.

I'll see if I can make this local buffer size configurable at run time. Default (16k per TLSv1 spec) is way too large, we don't have enough RAM to handle that.

Workaround: try to configure max TLS record size on you server to be less than 4k. https://www.igvita.com/2013/10/24/optimizing-tls-record-size-and-buffering-latency/

igrr commented 9 years ago

@timkay I have implemented Maximum Fragment Length Negotiation extenstion in my axTLS build. Sadly though, it seems that server doesn't support it and sends large records anyway.

timkay commented 9 years ago

For the record, firebaseio.com isn't my server, it is a service provided by Google. Firebase is very popular among IoT users. For example, it is the back end used by Nest. It will be a shame if we can't figure out a way for the ESP8266 to talk to it.

I will reach out to them to see if they can reconfigure at their end... I posted a question to Stack Overflow.

N0TB0T commented 9 years ago

Just curious:

Did anybody manage to use SSL with parse.com? It's basically used the same way as firebaseio, besides mostly for apps...

I did not get a call through, unfortunately.

cottsak commented 9 years ago

Can someone point me in the direction of doco or something that I can use to begin learning how to interpret crash dumps?


 ets Jan  8 2013,rst cause:2, boot mode:(3,6)

load 0x4010f000, len 1264, room 16 
tail 0
chksum 0x42
csum 0x42
~ld

I have no idea what this means and anything would be useful at this point. Thanks

igrr commented 9 years ago

Output posted by @grahamehorner indicates that there is an out-of-RAM condition. @cottsak the ets Jan 8 2013 (and all the following) stuff isn't related to the crash, so just ignore it. The stack dump you have posted above is valuable. Could you upload your compiled sketch (called iforgottocloseit.cpp.elf) somewhere for analysis?

Edit: i gave your sketch a shot, but it seems i need an account with Twilio to actually try anything.

sabas1080 commented 9 years ago

Support TLS 1.2?

igrr commented 9 years ago

@sabas1080 as mentioned a few comments above, axTLS only supports TLS 1.1

sabas1080 commented 9 years ago

Thanks @igrr , i have the next error in the API Telegram

IP address: 
192.168.1.65
GET Update Messages 
E:M 536

Soft WDT reset

ctx: cont 
sp: 3ffebc20 end: 3ffec1d0 offset: 01b0

stack>>>
3ffebdd0: 0f610597 21146ab8 e562ac99 4022c764

3ffebde0: ecfae377 c2e09b1a 694e65f8 3fff5710

3ffebdf0: 00000040 00000082 3fffa390 4022c9ee

3ffebe00: 3fffbc40 401000a2 00000000 00000041

3ffebe10: 0002df36 0000003f 00000000 00000041

3ffebe20: 7b70a77e 00000200 3fffa3b0 3fff5838

3ffebe30: 3fffbd4c ffffffff 3fff5710 3fff5710

3ffebe40: 00000040 3fff5710 3fffa390 3fff5710

3ffebe50: 00000040 3fff5710 3fffa390 4022d556

3ffebe60: 3fff57d8 0000003f cb13b5ec 00000020

3ffebe70: 3fff5320 3fff5710 3fff5858 00000020

3ffebe80: 3fff5320 3fff5710 00000010 4022d83c

3ffebe90: 3fff5878 3fff5818 3fff5858 00000001

3ffebea0: 00000001 3fff5858 00000000 4022c608

3ffebeb0: 00000100 3fff73eb 3fff73eb 00000000

3ffebec0: 00000100 3fff73eb 3fff57a0 4022e95c

3ffebed0: 00000000 000000d0 3ffebf00 00000030

3ffebee0: 23d24220 94505eb8 343d9c13 00000004

3ffebef0: 3fff87b8 3fff73b8 3fff73e5 4022a9b2

3ffebf00: d18f0203 5a0de732 a7a86b81 5a5fcdac

3ffebf10: dea207fb c0d93270 5df54da1 cdfe6d0d

3ffebf20: 5b8ad0c5 e16be47e bc1c8785 c0bc4001

3ffebf30: 3fff73e5 3fff87b8 3fff73b8 4022ad45

3ffebf40: f2fedcf8 0187f7fa 8c83b36b 3fff5228

3ffebf50: 00000000 00000004 00000004 40204dfb

3ffebf60: 3fff73e9 3fff73e5 3fff53b8 00001240

3ffebf70: 000012a0 3fff8667 0000001d 00000004

3ffebf80: 3fff73b8 3fff73e5 00000004 00000004

3ffebf90: 3fff73e5 3fff87b8 3fff73b8 4022a7da

3ffebfa0: 00000038 00000000 3fff5228 40204c56

3ffebfb0: 00000038 00000001 00000033 3fff73e0

3ffebfc0: 00000000 00020010 0000000e 00000010

3ffebfd0: 00000004 401000a2 3ffeae68 40204cb4

3ffebfe0: 3fff87b8 00000033 00000016 00000000

3ffebff0: 3fff87b8 3fff73b8 3fff73b8 4022a916

3ffec000: 3fff87b8 3fff73e5 3fff73b8 4022aaed

3ffec010: 00000000 3fff5280 3fff73b8 4022ab76

3ffec020: 3fff6d98 40205e8e 3ffeae68 3ffeb0e0

3ffec030: 3fff52f8 3fff5260 3ffeae68 40204ab6

3ffec040: c8a79a95 3ffe9701 402017ec 3ffe9701

3ffec050: 3ffec130 000001bb 3ffeae68 40204d28

3ffec060: 3ffe97e0 c8a79a95 00000000 40207afa

3ffec070: 3ffe9701 000001bb 3ffeae68 402032f5

3ffec080: 3ffe97e0 c8a79a95 3ffe97e0 c8a79a95

3ffec090: 3ffe95b6 3ffec154 3ffec154 40207bd0

3ffec0a0: 3ffe9701 3ffec154 3ffeae68 40204d40

3ffec0b0: 401000b4 3ffec130 3ffea8a4 40205426

3ffec0c0: 3fff4e98 401000a2 3ffec130 00000044

3ffec0d0: 00000043 00000001 3ffec130 40207afa

3ffec0e0: 3fff4e38 3fff4f28 0000000f 00000000

3ffec0f0: 3fff4f08 0000000f 00000002 40207b82

3ffec100: 3ffec238 3ffea8a4 3ffec130 40207c84

3ffec110: 3ffec238 3ffea8a4 00000000 3ffec1a0

3ffec120: 3ffec238 3ffea8a4 00000000 402055ab

3ffec130: 3fff4ea8 0000004f 00000044 00000001

3ffec140: 3ffec238 00000000 3ffe92dc 40207afa

3ffec150: 3fff3c98 3fff4f48 0000000f 00000000

3ffec160: 3fff4e38 0000004f 00000044 40207b82

3ffec170: 3fffdc20 00000000 3ffec1a0 40207c84

3ffec180: 4101a8c0 00ffffff 3ffea8a0 3ffec1fc

3ffec190: 3fffdc20 00000000 3ffea8a0 402022ca

3ffec1a0: 3fff4b78 0000000f 00000001 00000000

3ffec1b0: 00000000 00000000 3ffec1f4 4020186a

3ffec1c0: 00000000 00000000 3ffeb1b0 40100398

<<<stack<<<
ets Jan 8 2013,rst cause:2, boot mode:(3,6)

load 0x4010f000, len 1264, room 16 
tail 0
chksum 0x42
csum 0x42
~ld

I think is an out-of-RAM condition can you help me a little?

cottsak commented 9 years ago

@igrr Refer to the section Creating a Twilio account so you can send sms to get Twilio API creds. In the mean time, I'll get you that compiled sketch too.

geovanisouza92 commented 9 years ago

+1

igrr commented 9 years ago

@cottsak I got myself an account and changed your sketch to send an SMS every 10 seconds. 50 requests done, no crashes so far. Will leave it running.

cottsak commented 9 years ago

@igrr Perhaps there is something else different that we're doing.

Can you post the code you're using now so I can try it too? Also, what method are you using to push that code to esp? I'm just using Arduino IDE - could there be some subtle differences there that might make a difference?

tamberg commented 9 years ago

@abl re FOSS license exception: has been removed, PolarSSL/mbed is now available under the Apache 2.0 license. See https://tls.mbed.org/foss-license-exception

Sequoia commented 9 years ago

Sorry to add noise but this thread is pretty long, is there a wiki page or something entitled "How to Make HTTPS Requests Using the ESP8266 Arduino IDE" that steps thru setting this up? Sounds somewhat complex with the arduino IDE recompile/board management etc.

As a side-question: I use the Sparkfun Thing with their board definition-- will it be terribly complicated to get this code to run on that board? thanks.

cottsak commented 9 years ago

@Sequoia This tutorial I just published could help you out: http://hammerproject.com/post/130804023369/iot-intro-sms-me-when-i-leave-my-garage-door-open

whyameye commented 9 years ago

@mtnbrit it turns out the SSL library supports MQTT. Packets will be sent encrypted and server authenticity can be verified. However, as far as I know, we don't yet support client verification from the server.

To get MQTT to work with TLS, two lines need to be added in https://github.com/esp8266/Arduino/blob/master/libraries/ESP8266WiFi/src/WiFiClientSecure.cpp

for function

uint8_t WiFiClientSecure::connected() {
    if (_client->state() == ESTABLISHED)
        return 1;

    if (!_ssl)
        return 0;

    return _ssl->available() > 0;
}

add at the beginning of the function (before if (_client->state() == ESTABLISHED))

    if (!_client)
      return 0;

(@igrr should these lines be pushed into the git repo? I believe it's a mistake that it isn't already there.)

After that it's just a matter of using WiFiClientSecure instead of WiFIClient in the mqtt_esp8266 example.

igrr commented 9 years ago

Yes, it's a mistake. Thanks for spotting.

On Sun, Nov 8, 2015, 06:41 whyameye notifications@github.com wrote:

@mtnbrit https://github.com/mtnbrit it turns out the SSL library supports MQTT. Packets will be sent encrypted and server authenticity can be verified. However, as far as I know, we don't yet support client verification from the server.

To get MQTT to work with TLS, two lines need to be added in

https://github.com/esp8266/Arduino/blob/master/libraries/ESP8266WiFi/src/WiFiClientSecure.cpp

for function

uint8_t WiFiClientSecure::connected() { if (_client->state() == ESTABLISHED) return 1;

if (!_ssl)
    return 0;

return _ssl->available() > 0;

}

add at the beginning of the function (before if (_client->state() == ESTABLISHED))

if (!_client)
  return 0;

(@igrr https://github.com/igrr should these lines be pushed into the git repo? I believe it's a mistake that it isn't already there.)

After that it's just a matter of using WiFiClientSecure instead of WiFIClient in the mqtt_esp8266 example.

— Reply to this email directly or view it on GitHub https://github.com/esp8266/Arduino/issues/43#issuecomment-154773929.

sticilface commented 9 years ago

I'm trying to get my ESP to download from raw.githubusercontent.com, I get the handshake success, but then receive nothing. Is there a way for me to identify if it is due to the TLS fragment being too big?

igrr commented 9 years ago

@sticilface Yes you can uncomment DEBUGV in debug.h and DEBUG_TLS in WiFiClientSecure.cpp. Also add Serial.setDebugOutput(true); in setup() function.

sticilface commented 9 years ago

So here is the debug output. SNTP? 

Connected to raw.githubusercontent.com
pos:0 pos:3 pos:6 pos:9 pos:12 pos:15 pos:18 pos:21 pos:24 pos:27 pos:30 pos:33 pos:36 pos:39 pos:42 pos:45 pos:48 pos:51 pos:54 pos:57 certificate matches
please start sntp first !
:wr

:sent 277

:ww

GET /sticilface/ESPmanager/fixcrashing/examples/Settingsmanager-example/data/jquery.mobile-1.4.5.min.js.gz HTTP/1.1
Host: raw.githubusercontent.com
User-Agent: BuildFailureDetectorESP8266
Accept: */*
Connection: close

Waiting for server response: 
Recieved data:
Recieve end
File /jquery.mobile-1.4.5.min.js.gz 0 Bytes
please start sntp first !
:wr

:sent 53

:ww

:ur 1

:close
sticilface commented 9 years ago

Last debug msg is not so clear as I'm download 3 files back to back. here is one debug output from start to end.

===================== START ===================
File Created
HOST: raw.githubusercontent.com:443
:ref 1

please start sntp first !
please start sntp first !
:wr

:sent 56

:rn 1440

:ww

:rd 5, 1440, 0

:rdi 1440, 5

:rd 74, 1440, 5

:rdi 1435, 74

:rd 5, 1440, 79

:rdi 1361, 5

:rd 1356, 1440, 84

:rdi 1356, 1356

:c0 1356, 1440

:rn 1320

:rd 1311, 1320, 0

:rdi 1320, 1311

:rd 5, 1320, 1311

:rdi 9, 5

:rd 4, 1320, 1316

:rdi 4, 4

:c0 4, 1320

please start sntp first !
please start sntp first !
:wr

:sent 267

:ww

:wr

:sent 6

:ww

please start sntp first !
:wr

:sent 69

:ww

:rn 75

:rd 5, 75, 0

:rdi 75, 5

:rd 1, 75, 5

:rdi 70, 1

:rd 5, 75, 6

:rdi 69, 5

:rd 64, 75, 11

:rdi 64, 64

:c0 64, 75

Connected to raw.githubusercontent.com
pos:0 pos:3 pos:6 pos:9 pos:12 pos:15 pos:18 pos:21 pos:24 pos:27 pos:30 pos:33 pos:36 pos:39 pos:42 pos:45 pos:48 pos:51 pos:54 pos:57 certificate matches
please start sntp first !
:wr

:sent 261

:ww

Waiting for server response: 
File /jquery-1.11.1.min.js.gz 0 Bytes
please start sntp first !
:wr

:rn 677

:rch 677, 53

:rcla

:abort

:ww

:ur 1

SPIFFS_close: fd=1

SPIFFS_close: fd=1

/jquery-1.11.1.min.js.gz has been downloaded
N0TB0T commented 9 years ago

I'm now getting completely lost. @igrr I can't find any of the comments you mentioned in "uncomment DEBUGV in debug.h and DEBUG_TLS in WiFiClientSecure.cpp" - sry

Also, could anybody tell me, whether it makes any sense to investigate how to use parse.com with esp8266 at all? Two people seem to have had success connecting, but maybe that is bullshit.

Maybe anybody could have a look at the certificate of api.parse.com and tell me whether that's supposed to work?

Spending my fourth day now, managed to get the certificate accepted, but did not get one call through... PLZ!

sticilface commented 9 years ago

Sorry for posting yet another debug output, but i forgot to hit save when adding debugging to the secure client lib. so i have more debugging. hopefully this is useful to someone (probably only @igrr) I'm curious about the sntp thing. I've tried executing this from loop and not setup, but no change. also there is a huge drop in heap something like 7-8k, when the secureclient is used, and this occurs even it if is dynamic... using new. Are the secure client functions all held in RAM is there anyway to have them loaded from flash? (do forgive me if i'm talking out of my tree, and don't understand what is going on).

thanks for all the hard work igrr. 

===================== START ===================
:urch 68, 57

:urd 19, 57, 13

File Created
:urch 57, 127

HOST: raw.githubusercontent.com:443
:ref 1

please start sntp first !
please start sntp first !
State:  sending Client Hello (1)
:wr

:sent 56

:rn 1418

:ww

:rd 5, 1418, 0

:rdi 1418, 5

:rd 74, 1418, 5

:rdi 1413, 74

State:  receiving Server Hello (2)
:rd 5, 1418, 79

:rdi 1339, 5

:rd 1334, 1418, 84

:rdi 1334, 1334

:c0 1334, 1418

:rn 1342

:rd 1333, 1342, 0

:rdi 1342, 1333

State:  receiving Certificate (11)
:rd 5, 1342, 1333

:rdi 9, 5

:rd 4, 1342, 1338

:rdi 4, 4

:c0 4, 1342

State:  receiving Server Hello Done (14)
please start sntp first !
please start sntp first !
State:  sending Client Key Exchange (16)
:wr

:sent 267

:ww

:wr

:sent 6

:ww

State:  sending Finished (16)
please start sntp first !
:wr

:sent 69

:ww

:rn 75

:rd 5, 75, 0

:rdi 75, 5

:rd 1, 75, 5

:rdi 70, 1

:rd 5, 75, 6

:rdi 69, 5

:rd 64, 75, 11

:rdi 64, 64

:c0 first !
:wr

 64, 75

State:  receiving Finished (16)
Connected to raw.githubusercontent.com
pos:0 pos:3 pos:6 pos:9 pos:12 pos:15 pos:18 pos:21 pos:24 pos:27 pos:30 pos:33 pos:36 pos:39 pos:42 pos:45 pos:48 pos:51 pos:54 pos:57 certificate matches
please start sntp:sent 261

:ww

Waiting for server response: 
File /jquery-1.11.1.min.js.gz 0 Bytes
please start sntp first !
:wr

:sent 53

:ww

Alert: close notify
:ur 1

:close

SPIFFS_close: fd=1

SPIFFS_close: fd=1

/jquery-1.11.1.min.js.gz has been downloaded
Juppit commented 9 years ago

This sntp stuff comes probably in since https://github.com/esp8266/Arduino/commit/4cf72e7ef4fcabde230de44ba1d1ab5d57cdeee0

Do you use any time functions or includes time.h ?

On 13.11.2015 at 15:04 wrote sticilface:

Sorry for posting yet another debug output, but i forgot to hit save when adding debugging to the secure client lib. so i have more debugging. hopefully this is useful to someone (probably only @igrr https://github.com/igrr) I'm curious about the sntp thing. I've tried executing this from loop and not setup, but no change. also there is a huge drop in heap something like 7-8k, when the secureclient is used, and this occurs even it if is dynamic... using new. Are the secure client functions all held in RAM is there anyway to have them loaded from flash? (do forgive me if i'm talking out of my tree, and don't understand what is going on).

thanks for all the hard work igrr.

|===================== START =================== :urch 68, 57 :urd 19, 57, 13 File Created :urch 57, 127 HOST: raw.githubusercontent.com:443 :ref 1 please start sntp first ! please start sntp first ! State: sending Client Hello (1) :wr :sent 56 :rn 1418 :ww :rd 5, 1418, 0 :rdi 1418, 5 :rd 74, 1418, 5 :rdi 1413, 74 State: receiving Server Hello (2) :rd 5, 1418, 79 :rdi 1339, 5 :rd 1334, 1418, 84 :rdi 1334, 1334 :c0 1334, 1418 :rn 1342 :rd 1333, 1342, 0 :rdi 1342, 1333 State: receiving Certificate (11) :rd 5, 1342, 1333 :rdi 9, 5 :rd 4, 1342, 1338 :rdi 4, 4 :c0 4, 1342 State: receiving Server Hello Done (14) please start sntp first ! please start sntp first ! State: sending Client Key Exchange (16) :wr :sent 267 :ww :wr :sent 6 :ww State: sending Finished (16) please start sntp first ! :wr :sent 69 :ww :rn 75 :rd 5, 75, 0 :rdi 75, 5 :rd 1, 75, 5 :rdi 70, 1 :rd 5, 75, 6 :rdi 69, 5 :rd 64, 75, 11 :rdi 64, 64 :c0 first ! :wr 64, 75 State: receiving Finished (16) Connected to raw.githubusercontent.com pos:0 pos:3 pos:6 pos:9 pos:12 pos:15 pos:18 pos:21 pos:24 pos:27 pos:30 pos:33 pos:36 pos:39 pos:42 pos:45 pos:48 pos:51 pos:54 pos:57 certificate matches please start sntp:sent 261 :ww Waiting for server response: File /jquery-1.11.1.min.js.gz 0 Bytes please start sntp first ! :wr :sent 53 :ww Alert: close notify :ur 1 :close SPIFFS_close: fd=1 SPIFFS_close: fd=1 /jquery-1.11.1.min.js.gz has been downloaded |

— Reply to this email directly or view it on GitHub https://github.com/esp8266/Arduino/issues/43#issuecomment-156439322.

igrr commented 9 years ago

Time functions are used by SSL internally. However there is no issue with SNTP not being initialized. @sticilface this thread got quite long, did you post the sketch which corresponds to this debug output somewhere? Could you please link a gist or something?

igrr commented 9 years ago

@N0TB0T i went to parse.com but honestly I'm completely lost. What is the minimal amount of stuff i need to do to try their API?

N0TB0T commented 9 years ago

@igrr It's pretty straightforward: Get an Account (free), click "new app", done. You can find your app ID and keys under settings: https://www.parse.com/apps/yourAppName/edit#keys . From there on, it's just using their REST API, using the app-ID and the REST-API-Key... All data sent there via POST in json-format will show up at https://www.parse.com/apps/yourAppName/collections - here are the example calls: https://www.parse.com/docs/rest/guide

Thanks for trying!

N0TB0T commented 9 years ago

@igrr code on the clientside would look somewhat like this:

    client.print("POST /1/classes/yourAppName HTTP/1.1\r\n");
    client.print("Host: api.parse.com\r\n");
    client.print("Connection: close\r\n");
    client.print("Content-Type: application/json\r\n");
    client.print("User-Agent: ESP8266\r\n");
    client.print("X-Parse-Application-Id: ");
    client.print(PARSE_APP_ID);
    client.print("\r\n");
    client.print("X-Parse-REST-API-Key: ");
    client.print(PARSE_REST_API_KEY);
    client.print("\r\n");

    client.print("\"temperature\":17.4,\"temperatureReference\":17,\"humidity\":85,\"pressure\":1038.1,\"brightness\":40,\"batteryLevel\":97}");

At least I hope so.

sticilface commented 9 years ago

@igrr https://gist.github.com/sticilface/3795d4e2b8047c729bbe

igrr commented 9 years ago

@sticilface it seems to me that you need to replace SecClient.connected with SecClient.available at line 101. At this point you are connected because you just did SecClient.print a few lines above. On the other hand, data may not be available yet. SecClient.connected check passes immediately and then you hit while (SecClient.available()), and you miss this loop because no data is available.

igrr commented 9 years ago

@N0TB0T Thanks for info, I'll try on the weekend.

sticilface commented 9 years ago

ah, i did actually have it that way round to start with... but you make total sense. However, still no jo https://gist.github.com/sticilface/24842312767a2ed1fe1a

the clue might be this

wr

:er -9 53 1

:ww
igrr commented 9 years ago

@sticilface this config.htm file is pretty large, so yes, you are hitting max fragment size limitation.

---------------
Waiting for server response: 
---- BODY ---
:rd 5, 1452, 0
:rdi 1452, 5
Error: invalid protocol message

Final line is debug output from axTLS (i have uncommented #define DEBUG_SSL line in WiFiClientSecure.cpp).

sticilface commented 9 years ago

@igrr ok, thanks for that. Have to think of another solution to that. I guess HTTP hosting somewhere else