HTTPS server and Google Chrome

hmckinlay commented 2 years ago

Does this actually work for anyone on any hardware platforms?

Running the latest azure rtos & netxduo on a NUCLEO-H753ZI here. UDP, TCP, SNTP, DNS, HTTP server all working fine.

Updating to 6.1.9 with the fixes in issues 48 and 50 gets past the chrome erroring with ERR_SSL_VERSION_OR_CIPHER_MISMATCH but the SSL library still bombs with NX_SECURE_TLS_NO_SUPPORTED_CIPHERS.

tstapko commented 2 years ago

I'm sorry that you're having issues. Unfortunately, the error NX_SECURE_TLS_NO_SUPPORTED_CIPHERS means that Chrome isn't presenting any ciphersuites that are supported in your HTTPS server configuration.

According to the TLS RFCs (https://datatracker.ietf.org/doc/html/rfc5246 for TLS 1.2, https://datatracker.ietf.org/doc/html/rfc8446 for TLS 1.3) all TLS implementations MUST implement TLS_AES_128_GCM_SHA256 for TLS 1.3 and TLS_RSA_WITH_AES_128_CBC_SHA for TLS 1.2 (see "Mandatory Ciphersuite" sections). Azure RTOS TLS supports these ciphersuites by default.

However, we've seen different browsers (and Chrome in particular) refuse to send the mandatory ciphersuites in the TLS ClientHello. You can verify this is the case with Wireshark (or similar) and observe the TLS ciphersuites provided by the browser.

Due to the fact that Azure RTOS TLS is designed for microcontroller applications, it doesn't have a broad set of cryptographic algorithms so browsers that don't adhere to the RFCs will have issues when connecting to an Azure RTOS HTTPS server.

You have a few options:

Try to configure your browser to send compatible ciphersuites
Find a different browser that respects the RFCs (e.g. Firefox)
Attempt to implement your own cryptographic routines to support the browser and configuration you've chosen (see https://docs.microsoft.com/en-us/azure/rtos/netx-duo/netx-secure-tls/chapter3#cryptography-in-netx-secure-tls).
Use a less-specialized TLS stack (e.g. OpenSSL) that supports a larger number of ciphers with the assumption that there will be a match with your browser.

Unfortunately, we can't control what browser manufacturers do with regard to ciphersuites and if they don't follow the RFCs there's no guarantee that the browser will be compatible with any particular TLS server.

hmckinlay commented 2 years ago

Thanks for the quick update! Glad to know it's not just me going crazy. A little surprised this hasn't been raised before.

I have found Firefox has the same problem. Internet explorer works.

I think you should prioritise making this work becuase:

The world has moved on from IE, and even Microsoft is now pushing it's Chrome based browser
For this product to gain traction, it will need a HTTPS server that works with the most popular browsers in the world
Even for cloud deployments, https is a good way to recover or troubleshoot access to devices.

tstapko commented 2 years ago

It's definitely been raised before but I'll raise it again (not up to me to decide). Our primary focus for Azure RTOS is on being a TLS/HTTPS Client (device-to-cloud) and it's difficult keeping the TLS server up to date with whatever decisions the browser distributors make about TLS support. Azure RTOS TLS isn't a web-focused stack like OpenSSL and many devices we target don't have the resources to have both a cloud connection and run an HTTPS server anyway so it's likely that browser compatibility will remain somewhat limited. That being said, I will try to push get some of the Chrome-favored ciphers added to hopefully close the gap a little.

I will point out that Edge and Chrome are both based on Chromium (open-source browser core functionality) - ciphersuite choice is made by the distributor (Microsoft or Google) and not something intrinsic to the browser core so there's no technical reason for Edge and Chrome to offer the same ciphers.

hwmaier commented 2 years ago

Hi, I like to chime in here as we experienced this issue as well. We solved it by setting NX_SECURE_ENABLE_AEAD_CIPHER which resulted our NetXDuo server working with Chrome 93.

I do agree with @hmckinlay that a web server offering makes only sense if the typically used browsers can access it. Regardless whether Chrome follows RFC or not, they dominate at present the market.

If resources are the problem, then some of the outdated cybersuites could be removed or become optional. Also the uC market is moving to devices with larger memories like 1MB RAM and 2MB flash so these high end devices would be suitable to offer HTTPS.

From my observations encrypted networking will become more and more a standard requirement so HTTPS support will become mandatory.

hmckinlay commented 2 years ago

Hi, I like to chime in here as we experienced this issue as well. We solved it by setting NX_SECURE_ENABLE_AEAD_CIPHER which resulted our NetXDuo server working with Chrome 93.

I do agree with @hmckinlay that a web server offering makes only sense if the typically used browsers can access it. Regardless whether Chrome follows RFC or not, they dominate at present the market.

If resources are the problem, then some of the outdated cybersuites could be removed or become optional. Also the uC market is moving to devices with larger memories like 1MB RAM and 2MB flash so these high end devices would be suitable to offer HTTPS.

From my observations encrypted networking will become more and more a standard requirement so HTTPS support will become mandatory.

Good to know. I have tried this and it does not work for me with the latest Chrome 96. Does it work for you with the latest Chrome?

hmckinlay commented 2 years ago

What is the likelyhood that when the TLS 1.3 issues are resolved, Chrome will also work ?

hwmaier commented 2 years ago

Good to know. I have tried this and it does not work for me with the latest Chrome 96. Does it work for you with the latest Chrome?

Just tried again and I can confirm that latest Chrome 96.0.4664.110 is able to connect to the NetXDuo 6.1.9 nx_web_http_server component using these nx_user.h settings:

//
// NetX Secure.
//
// TLS 1.2 is enabled by default, TLS 1.0 and v1.1 are deprecated as are all SSL versions.
// #define NX_SECURE_TLS_ENABLE_TLS_1_3
// #define NX_SECURE_TLS_DISABLE_PROTOCOL_VERSION_DOWNGRADE
#define NX_SECURE_TLS_DISABLE_CLIENT_INITIATED_RENEGOTIATION
#define NX_SECURE_ALLOW_SELF_SIGNED_CERTIFICATES
// #define NX_SECURE_KEY_CLEAR // Slightly faster if diasabled by 1%
// #define NX_CRYPTO_ENABLE_UNALIGNED_ACCESS // Marginally faster by 0.3% but compilation error w/ FSP 3.4.0
#define NX_SECURE_ENABLE_AEAD_CIPHER // To enable connection with Chrome 93
// #define NX_SECURE_POWER_ON_SELF_TEST_MODULE_INTEGRITY_CHECK
#define NX_SECURE_TLS_CLIENT_DISABLED

I am using a self signed cert atm, so you will get NET::ERR_CERT_COMMON_NAME_INVALID but you can bypass this in Chrome by clicking on Advanced and then Proceed to 192.168.56.10 (unsafe). Once our product gets into production we will obtain a proper CA certificate.

hwmaier commented 2 years ago

I also tried using TLS 1.3 with these changed settings based on above settings:

#define NX_SECURE_TLS_ENABLE_TLS_1_3
#define NX_SECURE_TLS_DISABLE_PROTOCOL_VERSION_DOWNGRADE

and strangely I get this error ERR_TLS13_DOWNGRADE_DETECTED in Chrome. I have not investigated but surprised to see this despite setting NX_SECURE_TLS_DISABLE_PROTOCOL_VERSION_DOWNGRADE. I have the suspicion that NX_SECURE_TLS_DISABLE_PROTOCOL_VERSION_DOWNGRADE has not effect.

hwmaier commented 2 years ago

According to the TLS RFCs (https://datatracker.ietf.org/doc/html/rfc5246 for TLS 1.2, https://datatracker.ietf.org/doc/html/rfc8446 for TLS 1.3) all TLS implementations MUST implement TLS_AES_128_GCM_SHA256 for TLS 1.3 and TLS_RSA_WITH_AES_128_CBC_SHA for TLS 1.2 (see "Mandatory Ciphersuite" sections). Azure RTOS TLS supports these ciphersuites by default.

@tstapko Just reading your comment again. Chrome is in fact offering TLS_RSA_WITH_AES_128_CBC_SHA. But the _nx_crypto_ciphersuite_lookup_table does not contain TLS_RSA_WITH_AES_128_CBC_SHA. It contains TLS_RSA_WITH_AES_128_CBC_SHA256 instead.

tstapko commented 2 years ago

Ah, yes I forgot about that. You are correct - Microsoft security policy requires us to disable SHA-1 ciphersuites by default so we aren't technically compliant with the RFC for TLS 1.2 out of the box. You would have to create your own entry in the ciphersuite lookup table for TLS_RSA_WITH_AES_128_CBC_SHA

tstapko commented 2 years ago

As for the TLS 1.3 downgrade error, I'm not sure exactly what the issue is you're seeing, but my guess is that Chrome isn't providing an ECC curve that Azure RTOS TLS supports and then it attempts to downgrade to TLS 1.2 because it can't connect with 1.3.

tstapko commented 2 years ago

What is the likelyhood that when the TLS 1.3 issues are resolved, Chrome will also work ?

It's unlikely that fixing the TLS 1.3 issues would change anything unless you are using RSA certificates in your server - the issue there is that TLS 1.3 currently only supports ECC certs (a known issue we are aiming to fix in an upcoming release). If you have an ECC cert now and it doesn't work, that fix won't change the behavior.

TiejunMS commented 2 years ago

Close as no response.

hmckinlay commented 1 year ago

Does anyone know if there have been any updates on this in the last year? Thanks!

hwmaier commented 1 year ago

Does anyone know if there have been any updates on this in the last year? Thanks!

@hmckinlay I initially was quite interested to have Chrome working with HTTPS but since my last post here, I have not continued with NetxDuo as the MCU I wanted to use is not available for the foreseeable future, so development has been shelved.

yanwucai commented 1 year ago

@hmckinlay I tested it with Chrome and it is working when NX_SECURE_ENABLE_AEAD_CIPHER is defined. Can you take a look at the nx_crypto_tls_ciphers that is used for the nx_web_http_server_secure_configure function and check if TLS_RSA_WITH_AES_128_GCM_SHA256 is included in the _nx_crypto_ciphersuite_lookup_table?

I also tested it with TLS 1.3 and it works only when ECC certificates are used for the HTTPS server.

hmckinlay commented 1 year ago

Just got back to this after a long break, was hoping this would be fixed with time, sadly the problems still persist. Right now working on a -STM32F7XX and STM32H7XX -STM32CubeMx and STM32CubeIDE -Azure® RTOS release 6.1.10-12 -TLS 1.2 or 1.3 -RSA or ECC certs

Chrome erroring with ERR_CONNECTION_RESET. Firefox errors with PR_CONNECT_RESET_ERROR. Internet Explorer works.

How exactly would i add TLS_RSA_WITH_AES_128_CBC_SHA in the lookup table? I have tried to do this but end up with the same result.

hmckinlay commented 1 year ago

Can we please re-open this case? This problem is preventing us from moving to this product =(

yanwucai commented 1 year ago

If you want to add the SHA1 cipher suite, you can add the following line in the lookup table: { TLS_RSA_WITH_AES_128_CBC_SHA, &crypto_method_rsa, &crypto_method_rsa, &crypto_method_aes_cbc_128, 16, 16, &crypto_method_hmac_sha1, 20, &crypto_method_tls_prf_sha_256 },

For the connection reset problem, can you get the corresponding error code from NetX Secure?

hmckinlay commented 1 year ago

Using STM32 dev environment

NUCLEO-H753ZI

STM32CubeIDE 1.1.12

STM32CubeMX 6.7.0

NetxDuo Version 6.1.12

is assume you mean: -NX_SECURE_TLS_CIPHERSUITE_INFO _nx_crypto_ciphersuite_lookup_table[] in nx_crypto_generic_ciphersuites.c if you are using RSA cert and TLS 1.2

hash size 0x20 (32) rather than 20
crypto_method_tls_prf_sha256 rather than crypto_method_tls_prf_sha_256

I have tried lots of different combo's of certificates, ciphers, NETXDUO versions, none of which have worked. Right now using RSA cert and TLS 1.2 and currently getting ERR_CONNECTION_RESET, then after a while ERR_CONNECTION_TIMED_OUT.

before after adding the above:

ping to the device and and telnet on port 443 works at startup
chrome in incognito mode connection just times out
code does not reach _nx_secure_tls_server_handshake(4 params) in nx_secure_tls_server_handshake.c
after trying to connect in chrome ping and telnet no longer work.

after adding the above:

ping to the device and and telnet on port 443 works at startup
chrome in incognito mode connection just keeps retrying, SSL cert 'not secure' appears (one step forward!)
browser retries several times and repeats through _nx_secure_tls_server_handshake(4 params) in nx_secure_tls_server_handshake.c and returns NX_SUCCESS
after a while ERR_CONNECTION_RESET turns to ERR_CONNECTION_TIMED_OUT ping and telnet no longer work.

Which other part(s) of NetX Secure would you like a return code from?

This code is very time consuming to debug, it really needs some sort of logging. And for the https server to work of course.

yanwucai commented 1 year ago

The crypto method for the PRF is crypto_method_tls_prf_sha256, but the hash size for SHA1 should be 20. {TLS_RSA_WITH_AES_128_CBC_SHA, &crypto_method_rsa, &crypto_method_rsa, &crypto_method_aes_cbc_128, 16, 16, &crypto_method_hmac_sha1, 20, &crypto_method_tls_prf_sha256}, If you are using nx_crypto_tls_ciphers, then add this line to _nx_crypto_ciphersuite_lookup_table. If you are using nx_crypto_tls_ciphers_ecc, you can add this line and the following to _nx_crypto_ciphersuite_lookup_table_ecc.

{TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, &crypto_method_ecdhe, &crypto_method_ecdsa, &crypto_method_aes_cbc_128, 16, 16, &crypto_method_hmac_sha1, 20, &crypto_method_tls_prf_sha256}, {TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, &crypto_method_ecdhe, &crypto_method_rsa, &crypto_method_aes_cbc_128, 16, 16, &crypto_method_hmac_sha1, 20, &crypto_method_tls_prf_sha256},

If the 'not secure' appears, the cipher suite is accepted by Chrome. The problem looks like there are packet leaks somewhere. Could you please log the available pool count of the packet pools before or after each connection and see if it will drop to 0. For example printf("%lu, %lu", NxAppPool.nx_packet_pool_available, WebServerPool.nx_packet_pool_available)

bo-ms commented 11 months ago

Closing, no further response.

robert-kau commented 3 months ago

Is there any update on this? I have the same problem with Chrome.

I am also trying to use a self-signed certificate to set up an HTTPS server, but I am having trouble loading the private RSA key: the function nx_secure_x509_certificate_initialize always returns the error NX_SECURE_PKCS1_INVALID_PRIVATE_KEY. Has anyone had this problem? How do you generate the key and certificate?

agrutter commented 3 months ago

I’m not connecting yet on my project, but I was able to get NX_SUCCESS from that specific function call.

For my project, I’m using Renesas Synergy Software Platform v2.6.0 and using their GUI configurator tool to configure NetX/ThreadX I enabled “Allow Self Signed Certificates” and the code generator updated my nx_secure_user.h to include #define NX_SECURE_ALLOW_SELF_SIGNED_CERTIFICATES.

I read through these instructions and example to determine my initialization function call sequence: https://github.com/eclipse-threadx/rtos-docs/blob/main/rtos-docs/netx-duo/netx-duo-web-http/chapter2.md

I used these instructions to generate everything: https://devcenter.heroku.com/articles/ssl-certificate-self?ref=caffeinecoding.com

I followed these instructions to convert the cert and key to DER format: https://wiki.segger.com/HOWTO_convert_PEM_certificates_and_keys_to_DER_format_for_emSSL

I’m still not connecting though, wireshark shows some failures in the TLS handshaking elsewhere but I haven’t finished debugging to see what my issue is, but I hope this is slightly helpful at least.

Alex Grutter Engineer Future Designs, Inc. 996 A Cleaner Way SW Huntsville, AL 35805 Mobile: (256) 609-8767 Phone: (256) 883-1240 ext. 34 Fax: (256) 883-1241

From: Robert Buss Kaufmann @.> Sent: Wednesday, July 17, 2024 8:30 AM To: eclipse-threadx/netxduo @.> Cc: Subscribed @.***> Subject: Re: [eclipse-threadx/netxduo] HTTPS server and Google Chrome (Issue #67)

Is there any update on this? I have the same problem with Chrome.

I am also trying to use a self-signed certificate to set up an HTTPS server, but I am having trouble loading the private RSA key: the function nx_secure_x509_certificate_initialize always returns the error NX_SECURE_PKCS1_INVALID_PRIVATE_KEY. Has anyone had this problem? How do you generate the key and certificate?

— Reply to this email directly, view it on GitHubhttps://github.com/eclipse-threadx/netxduo/issues/67#issuecomment-2233330730, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIBCG7MSYGU7KYHT3ECQ6ZTZMZWU5AVCNFSM6AAAAABLAUWHAGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMZTGMZTANZTGA. You are receiving this because you are subscribed to this thread.Message ID: @.**@.>>

robert-kau commented 3 months ago

Hi, @agrutter

Thanks for the response.

Even generating the certificate and key with the provided instructions, I am still facing the same problem. Following the initialization function, the point where it fails is here, where tlv_type is 16 and not 2, like NX_SECURE_ASN_TAG_INTEGER:

Can you tell me the version of your OpenSSL?

agrutter commented 3 months ago

I’m using the 1.1.1 that comes with the WSL Ubuntu. I was able to work through my remaining issues yesterday, but as I already mentioned, I didn’t have your specific issue.

If the cert isn’t the issue, maybe you could find something by comparing Wireshark traffic coming in with what is available in the buffers at the point of failure? Good luck!

Here is the specific command I used to create the cert that ultimately worked for me.

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout server.key -out server.crt -config config.cnf -extensions 'req_ext'

config.cnf: [req] distinguished_name = req_distinguished_name req_extensions = req_ext prompt = no

[req_distinguished_name] C = US ST = StateName L = CityName O = CompanyName OU = DepartmentName CN = www.commonname.comhttp://www.commonname.com

[req_ext] subjectAltName = @alt_names

[alt_names] DNS.1 = www.commonname.comhttp://www.commonname.com

Thanks,

Alex Grutter Engineer Future Designs, Inc. 996 A Cleaner Way SW Huntsville, AL 35805 Mobile: (256) 609-8767 Phone: (256) 883-1240 ext. 34 Fax: (256) 883-1241

From: Robert Buss Kaufmann @.> Sent: Wednesday, July 17, 2024 9:42 AM To: eclipse-threadx/netxduo @.> Cc: Alex Grutter @.>; Mention @.> Subject: Re: [eclipse-threadx/netxduo] HTTPS server and Google Chrome (Issue #67)

Hi, @agrutterhttps://github.com/agrutter

Thanks for the response.

Even generating the certificate and key with the provided instructions, I am still facing the same problem. Following the initialization function, the point where it fails is here, where tlv_type is 16 and not 2, like NX_SECURE_ASN_TAG_INTEGER: image.png (view on web)https://github.com/user-attachments/assets/d7322695-acfe-4ed6-b3a5-1173ca55a673

Can you tell me the version of your OpenSSL?

— Reply to this email directly, view it on GitHubhttps://github.com/eclipse-threadx/netxduo/issues/67#issuecomment-2233495510, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIBCG7LP7AV3HQ4NF7UP6ZTZMZ7DDAVCNFSM6AAAAABLAUWHAGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMZTGQ4TKNJRGA. You are receiving this because you were mentioned.Message ID: @.**@.>>

robert-kau commented 2 months ago

Thanks for the information. This information helped me find the source of the problem. I was using OpenSSL version 3.0, which generates the RSA key with a different and higher version than PKCS#1. I used the -traditional flag, including the conversion from PEM to DER, and it started working.

I'm just noticing a 5s delay in loading the http page with TLS, when compared to the connection without TLS. I'm investigating how to improve this.

Also, I'm trying to use an elliptic curve key, to use TLS 1.3.

hwmaier commented 2 months ago

I'm just noticing a 5s delay in loading the http page with TLS, when compared to the connection without TLS. I'm investigating how to improve this.

You would need hardware encryption to improve the connection speed. I am using a Renesas RA6M5 device and when using hardware encryption for TLS it takes about 148ms for the first page load which is reasonable. After that (assuming you keep connection alive) the next page loads within 2ms.

Hnz2 commented 2 months ago

Just to be mentioned. There are two types of hardware accelerators which can improve TLS performance.

RSA accelerator which can improve TLS establishing time (as was illustrated by @hwmaier)
AES accelerator which improve overall data throughput for TLS

Not all MCU are equipped by RSA accelerator. But almost all modern MCUs have AES accelerator. If MCU is not equipped by RSA accelerator to improve software RSA is important this: enabled compiler optimisation; use highest CPU clock as possible; if MCU have data cache (like Cortex-M7) make sure that data cache is enabled by MPU unit at RAM which is used as stack for thread which used TLS.

eclipse-threadx / netxduo

HTTPS server and Google Chrome #67