search

arduino-libraries / MKRGSM

GNU Lesser General Public License v2.1
55 stars 51 forks source link

GSMSSLClient: empty response #85

Closed victorh800 closed 5 years ago

victorh800 commented 5 years ago

edit: possibly root cause (pun intended) found. original issue left as is. please read follow up comment.

symptom

GSMSSLClient disconnects after returning an empty response during GET and POST operations to some valid server/path combinations. and I mean empty response, not even a non-200 status message, no nothing.

how to reproduce

on the GsmSSLWebClient example, replace line 35 char server[] = "arduino.cc"; with char server[] = "www.arduino.cc"; (note the additional www.). of course, also modify arduino_secrets.h according your SIM's APN, login and password.

you will get this response:

Starting Arduino web client.
connecting...
connected

disconnecting.

instead of the expected:

Starting Arduino web client.
connecting...
connected
HTTP/1.1 200 OK
Server: nginx
Date: Wed, 24 Apr 2019 16:53:59 GMT
Content-Type: text/plain
Content-Length: 2263
Last-Modified: Wed, 02 Oct 2013 13:46:47 GMT
Connection: close
Vary: Accept-Encoding
ETag: "524c23c7-8d7"
Accept-Ranges: bytes

[asciilogo.txt contents]

disconnecting.

remarks

maybe I'm not understanding something basic about HTTPS or the MKRGSM library itself but I don't get why only the (unmodified) example works with GSMSSLClient, considering that both URLs, https://arduino.cc/asciilogo.txt and https://www.arduino.cc/asciilogo.txt, are valid using a standard browser.

of course, my production code goes further than the example. it points to an API with an URL that I can't nor choose nor modify.

victorh800 commented 5 years ago

initially I assumed that both arduino.cc and www.arduino.cc had the same root cert. well, I just discovered they have completely differently certification paths! arduino cc www arduino cc

that could confirm my initial suspicion that the issue was caused for not having loaded the necessary (root) cert.

however this does not explain why GSMSSLClient still returns an empty response from my custom API server. that's because, for pure coincidence, its host has the same path of arduino.cc, the one that does work! so now I'm more confused than before. myapi and it confuses me more that "DST_Root_CA_X3" is included in GSMRootCerts.h. so the whole thing should work.

any words of wisdom will be appreciated. in the mean time I'll try to manually load the cert to the library.

sandeepmistry commented 5 years ago

Hi @victorh800,

I open pull request #87 to correct the return code of connect(...) on failure, it was returning 2 instead of 0.

Could you change the GSM gsmAccess; line in the sketch to GSM gsmAccess(true); to provide some debug logs?

in the mean time I'll try to manually load the cert to the library. Did you get this working?

victorh800 commented 5 years ago

I open pull request #87 to correct the return code of connect(...) on failure, it was returning 2 instead of 0.

thanks! I had that bug just discovered, it certainly delayed me while trying to debug this issue.

Could you change the GSM gsmAccess; line in the sketch to GSM gsmAccess(true); to provide some debug logs?

will do. I'll send logs in less than an hour. ⏳

in the mean time I'll try to manually load the cert to the library.

Did you get this working?

yeap. I did it using the SerialGSMPassthrough example. I double-checked that the cert loaded by the library (DST_Root_CA_X3) is the very same than the website I'm testing on.

victorh800 commented 5 years ago

there you go. each log is self-explanatory from its filename.

server=arduino.cc.txt server=www.arduino.cc.txt

I didn't know about the debug log of the GSM class before today, so i'll explore more of it tomorrow.

sandeepmistry commented 5 years ago

@victorh800 I'll prepare a pull request like https://github.com/arduino-libraries/MKRNB/pull/15 to add the "Starfield Services Root Certificate Authority - G2" root CA to this library.

sandeepmistry commented 5 years ago

hmm, a change like https://github.com/arduino-libraries/MKRNB/pull/15 doesn't seem to work on the MKR GSM 1400, @facchinm do you have any ideas on this?

victorh800 commented 5 years ago

@sandeepmistry, the problem is not with the certs. there is a bug in one of the AT commands. I isolated it today and I believe I solved it. right now I'm not in front of the code. I'll post my findings shortly.

sandeepmistry commented 5 years ago

cc/ @gvarisco

victorh800 commented 5 years ago

you see, according to the AT command manual (rev R62, 26.3.2) the syntax for +USECPRF is AT+USECPRF=<profile_id>[,<op_code>[,<param_val>]], meaning this command has, at most, three parameters. however the actual code, contains AT+USECPRF=0,0,1,4,<_host>, meaning five parameters. curiously, the modem issues an OK message (I tested it manually with SerialGSMPassthrough) in spite of being an ill-formed command. maybe this is the origin of the bug.

I can only speculate that the original developer meant this: AT+USECPRF=0,0,1 // op_code 0: certificate validation level AT+USECPRF=0,4,<_host> // op_code 4: expected server hostname

I tested this hypothesis doing a GET to "www.arduino.cc" and it worked as intended. I'll do a pull request with this modification so it can be tested.

sandeepmistry commented 5 years ago

I tested this hypothesis doing a GET to "www.arduino.cc" and it worked as intended. I'll do a pull request with this modification so it can be tested.

Can you open a pull request for your changes? I've updated pull request #88 to correct the AT command - however connecting to www.arduino.cc via GSMSSLClient still doesn't work for me.

sandeepmistry commented 5 years ago

Hi @victorh800,

I've opened pull request https://github.com/arduino-libraries/MKRGSM/pull/90 to resolve this. If you get a chance please try it out and provide your feedback. Thanks.