search

arduino-libraries / MKRGSM

GNU Lesser General Public License v2.1
55 stars 51 forks source link

SSL connection to specific website fails #124

Closed jwindhager closed 3 years ago

jwindhager commented 3 years ago

Hello

I'm trying to connect to https://mattermost.dqbm.uzh.ch using the GsmSSLWebClient example sketch, with GSM gsmAccess(true) for debugging. However, the connection fails at the USOCO AT command without further details:

connecting...
AT+USECMNG=0,0,"AddTrust_External_CA_Root",1082
>
+USECMNG: 0,0,"AddTrust_External_CA_Root","1d3554048578b03f42424dbf20730a3f"

OK
AT+USECMNG=0,0,"Baltimore_CyberTrust_Root",891
>
+USECMNG: 0,0,"Baltimore_CyberTrust_Root","acb694a59c17e0d791529bb19706a6e4"

OK
AT+USECMNG=0,0,"COMODO_RSA_Certification_Authority",1500
>
+USECMNG: 0,0,"COMODO_RSA_Certification_Authority","1b31b0714036cc143691adc43efdec18"

OK
AT+USECMNG=0,0,"DST_Root_CA_X3",846
>
+USECMNG: 0,0,"DST_Root_CA_X3","410352dc0ff7501b16f0028eba6f45c5"

OK
AT+USECMNG=0,0,"DigiCert_High_Assurance_EV_Root_CA",969
>
+USECMNG: 0,0,"DigiCert_High_Assurance_EV_Root_CA","d474de575c39b2d39c8583c5c065498a"

OK
AT+USECMNG=0,0,"Entrust_Root_Certification_Authority",1173
>
+USECMNG: 0,0,"Entrust_Root_Certification_Authority","d6a5c3ed5ddd3e00c13d87921f1d3fe4"

OK
AT+USECMNG=0,0,"Equifax_Secure_Certificate_Authority",804
>
+USECMNG: 0,0,"Equifax_Secure_Certificate_Authority","67cb9dc013248a829bb2171ed11becd4"

OK
AT+USECMNG=0,0,"GeoTrust_Global_CA",856
>
+USECMNG: 0,0,"GeoTrust_Global_CA","f775ab29fb514eb7775eff053c998ef5"

OK
AT+USECMNG=0,0,"GeoTrust_Primary_Certification_Authority_G3",1026
>
+USECMNG: 0,0,"GeoTrust_Primary_Certification_Authority_G3","b5e83436c910445848706d2e83d4b805"

OK
AT+USECMNG=0,0,"GlobalSign",958
>
+USECMNG: 0,0,"GlobalSign","9414777e3e5efd8f30bd41b0cfe7d030"

OK
AT+USECMNG=0,0,"Go_Daddy_Root_Certificate_Authority_G2",969
>
+USECMNG: 0,0,"Go_Daddy_Root_Certificate_Authority_G2","803abc22c1e6fb8d9b3b274a321b9a01"

OK
AT+USECMNG=0,0,"VeriSign_Class_3_Public_Primary_Certification_Authority_G5",1239
>
+USECMNG: 0,0,"VeriSign_Class_3_Public_Primary_Certification_Authority_G5","cb17e431673ee209fe455793f30afa1c"

OK
AT+USECMNG=2,0,"AmazonRootCA1"

ERROR
AT+USECMNG=0,0,"Starfield_Services_Root_Certificate_Authority_G2",1011
>
+USECMNG: 0,0,"Starfield_Services_Root_Certificate_Authority_G2","173574af7b611cebf4f93ce2ee40f9a2"

OK
AT+USOCR=6

+USOCR: 0

OK
AT+USOSEC=0,1,0

OK
AT+USECPRF=0,0,0

OK
AT+USOCO=0,"mattermost.dqbm.uzh.ch",443

ERROR

+UUSOCL: 0
AT+USOCL=0

ERROR
connection failed

At first, I assumed this could be a problem with the QuoVadis CA, so I added the respective root certificates using client.setUserRoots(...); at the beginning of setup(). This didn't work, so I tried replacing that command with client.setCertificateValidationLevel(0);, which didn't help either.

The board is well-powered with a fully charged 2400mAh LiPo and a suitable USB PSU. HTTPS requests to other sites (google.com, arduino.cc) work without any problem. Also, from experience, I would expect the board to hang if there is a problem related to the power supply.

One thing I noticed is that the SSL certificate for https://mattermost.dqbm.uzh.ch contains umlauts. Could this cause a problem with the SARA-U201? image

I also tried mimicking the request on my workstation:

❯ curl -v https://mattermost.dqbm.uzh.ch:443/ --http1.1 -H "User-Agent:" -H "Accept:"
*   Trying 130.60.168.73:443...
* TCP_NODELAY set
* Connected to mattermost.dqbm.uzh.ch (130.60.168.73) port 443 (#0)
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: C=CH; ST=Zrich; L=Zrich; O=Universitaet Zuerich; OU=Department of Quantitative Biomedicine; CN=mattermost.dqbm.uzh.ch
*  start date: Oct 16 11:44:08 2019 GMT
*  expire date: Oct 16 11:54:00 2021 GMT
*  subjectAltName: host "mattermost.dqbm.uzh.ch" matched cert's "mattermost.dqbm.uzh.ch"
*  issuer: C=BM; O=QuoVadis Limited; CN=QuoVadis Global SSL ICA G2
*  SSL certificate verify ok.
> GET / HTTP/1.1
> Host: mattermost.dqbm.uzh.ch
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Server: nginx
< Date: Tue, 29 Dec 2020 11:45:47 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 3091
< Connection: keep-alive
< Accept-Ranges: bytes
< Cache-Control: no-cache, max-age=31556926, public
< Content-Security-Policy: frame-ancestors 'self'; script-src 'self' cdn.rudderlabs.com
< Last-Modified: Thu, 17 Dec 2020 07:35:41 GMT
< X-Frame-Options: SAMEORIGIN
< X-Request-Id: swmj9wccm7b5urxwhe54p43pyw
< X-Version-Id: 5.29.0.5.29.0.0425a808c70766a6e00c06d342625f28.true
< Strict-Transport-Security: max-age=31536000; includeSubDomains
< 
...

I noticed that, without forcing HTTP/1.1 with --http1.1, the server responds with HTTP/2. Could this be a problem?

Thanks

jwindhager commented 3 years ago

Another URL that doesn't work: www.arduino.cc

The original URL in the example sketch (arduino.cc) responds with an HTTP 301 as expected. Other URLs, e.g. www.google.com, work as expected.

Again, adding the certificates using client.setUserRoots(...); doesn't help. Also tried to replace all USECPRF AT commands in GSMClient.cpp and GSMSSLClient.cpp with USECPRF=0, without success.

EDIT 1: Also checked for TLS version (all tested domains support TLSv1.2) and cipher algorithm (all use either ECDHE-ECDSA-CHACHA20-POLY1305 or ECDHE-RSA-AES256-GCM-SHA384, but no patterns there). Also tried to enforce TLS version and/or cipher suite using AT commands, without success. Factory-resetting the SARA-U201 also doesn't seem to do anything.

EDIT 2: Enabling advanced error output AT+CMEE=2 doesn't yield any additional information (only "ERROR" after the USOCO AT command, see above).

EDIT 3: This issue seems to be closely related to https://github.com/arduino-libraries/MKRGSM/issues/85, but in my case the certificate seems to be correctly stored on the modem already (compare with the [MD5 code obtained from the certificate PEM file](http://icyberchef.com/#recipe=PEM_to_Hex()From_Hex('None')MD5()&input=LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURkekNDQWwrZ0F3SUJBZ0lFQWdBQXVUQU5CZ2txaGtpRzl3MEJBUVVGQURCYU1Rc3dDUVlEVlFRR0V3SkoKUlRFU01CQUdBMVVFQ2hNSlFtRnNkR2x0YjNKbE1STXdFUVlEVlFRTEV3cERlV0psY2xSeWRYTjBNU0l3SUFZRApWUVFERXhsQ1lXeDBhVzF2Y21VZ1EzbGlaWEpVY25WemRDQlNiMjkwTUI0WERUQXdNRFV4TWpFNE5EWXdNRm9YCkRUSTFNRFV4TWpJek5Ua3dNRm93V2pFTE1Ba0dBMVVFQmhNQ1NVVXhFakFRQmdOVkJBb1RDVUpoYkhScGJXOXkKWlRFVE1CRUdBMVVFQ3hNS1EzbGlaWEpVY25WemRERWlNQ0FHQTFVRUF4TVpRbUZzZEdsdGIzSmxJRU41WW1WeQpWSEoxYzNRZ1VtOXZkRENDQVNJd0RRWUpLb1pJaHZjTkFRRUJCUUFEZ2dFUEFEQ0NBUW9DZ2dFQkFLTUV1eUtyCm1EMVg2Q1p5bXJWNTFDbmk0ZWlWZ0xHdzQxdU9LeW1hWk4raFhlMndDUVZ0MnlndXptS2lZdjYwaU5vUzZ6anIKSVozQVFTc0JVbnVJZDlNY2o4ZTZ1WWkxYWdubmMrZ1JRS2ZSek1waWpTM2xqd3VtVU5Lb1VNTW82dldySlllSwptcFljcVdlNFB3elY5L2xTRXkvQ0c5VndjUENQd0JMS0JzdWE0ZG5LTTNwMzF2anN1ZkZvUkVKSUU5TEF3cVN1ClhtRCt0cVlGL0xUZEIxa0MxRmtZbUdQMXBXUGdrQXg5WGJJR2V2T0Y2dXZVQTY1ZWhENWYveFh0YWJ6NU9UWnkKZGM5M1VrM3p5WkFzdVQzbHlTTlRQeDhrbUNGY0I1a3B2Y1k2N09kdWhqcHJsM1JqTTcxb0dESHdlSTEydi95ZQpqbDBxaHFkTmtOd25HamtDQXdFQUFhTkZNRU13SFFZRFZSME9CQllFRk9XZFdUQ0NSMWpNclBvSVZEYUdlenExCkJFM3dNQklHQTFVZEV3RUIvd1FJTUFZQkFmOENBUU13RGdZRFZSMFBBUUgvQkFRREFnRUdNQTBHQ1NxR1NJYjMKRFFFQkJRVUFBNElCQVFDRkRGMk81RzlSYUVJRm9OMjdUeWNsaEFPOTkyVDlMZGN3NDZRUUYrdmFLU20yZVQ5Mgo5aGtUSTdnUUN2bFlwTlJoY0wwRVlXb1NpaGZWQ3IzRnZEQjgxdWtNSlkyR1FFL3N6S04rT01ZM0VVL3QzV2d4CmprelNzd0YwN3I1MVhnZElHbjl3L3haY2hNQjVoYmdGL1grK1pSR2pEOEFDdFBoU056a0UxYWt4ZWhpL29DcjAKRXBuM28wV0M0enhlOVoyZXRjaWVmQzdJcEo1T0NCUkxiZjF3YldzYVk3MWs1aCszenZEeW55NjdHN2Z5VUloegprc0xpNHhhTm1qSUNxNDRZM2VrUUVlNStOYXVRcno0d2xIclFNejJuWlEvMS9JNmVZczlIUkN3Qlhic2R0VExTClI5STRMdEQrZ2R3eWFoNjE3anpWL09lQkhSbkRKRUxxWXptcAotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0t)):

AT+USECMNG=4,0,"Baltimore_CyberTrust_Root"
+USECMNG: 4,0,"Baltimore_CyberTrust_Root","acb694a59c17e0d791529bb19706a6e4"

EDIT 4: I ran the complete connection procedure as AT commands using the Serial Pass-Through sketch, and the problem persists. Suggests to me that this is a hardware problem / the specific host is incompatible with the u-blox modem for some reason?

jwindhager commented 3 years ago

Update: seems to be an issue with the modem's TLS firmware. I now use the non-SSL GSMClient with OPEnSLab-OSU's SSLClient and this works for my application. Also, I noticed that the modem's firmware is quite outdated. Unfortunately, u-blox doesn't publicly release firmware updates (not sure if this would help, though)...

jwindhager commented 3 years ago

Heard back from Arduino support:

www.arduino.cc was recently put behind a Cloudflare proxy, which requires more advanced TLS ciphers that are not supported by the default TLS stack of the GSM module which works with most servers and is the recommended choice. However, when it does not support a particular server, the BearSSL approach here can be used.

Looks like I did not check the proxy ciphers...