search

erlang / otp

Erlang/OTP
http://erlang.org
Apache License 2.0
11.39k stars 2.95k forks source link

TLS 1.3, ECC certificates and Java are not getting along #6435

Closed lukebakken closed 1 year ago

lukebakken commented 2 years ago

Describe the bug

Reported here: https://groups.google.com/g/rabbitmq-users/c/4qqaOO-QZf4

A user provided X509 ECC certificates (secp256r1/secp381r1 curves) that consistently cause Erlang to error with the following during the TLS handshake:

{error, {tls_alert, {handshake_failure, "TLS server: In state wait_cv at tls_handshake_1_3.erl:2071 generated SERVER ALERT: Fatal - Handshake Failure\n \"CertificateVerify uses unsupported signature algorithm\""}}}

Note that this only appears to happen with Java clients! If you test with openssl s_client (via run-openssl-client.sh in my repo), it works fine.

To Reproduce

I used Erlang 25.1.2 in all of my testing on Linux and Windows.

On Linux:

I have provided equivalent Powershell scripts for reproducing this issue on Windows.

Expected behavior

No TLS handshake errors.

Affected versions

$ openssl version OpenSSL 1.1.1q 5 Jul 2022

* Windows versions:

java -version openjdk version "19" 2022-09-20 OpenJDK Runtime Environment Temurin-19+36 (build 19+36) OpenJDK 64-Bit Server VM Temurin-19+36 (build 19+36, mixed mode, sharing)

openssl version OpenSSL 1.1.1q 5 Jul 2022

lukebakken commented 2 years ago

Granted, this could be a bug in Java but I thought I'd report it here just in case (cc @IngelaAndin)

I just added run-openssl-server* scripts that run openssl s_server. Running the ssltest.jar test application seems to work correctly.

lukebakken commented 2 years ago

I've added a couple additional test programs to my repo from this PR - https://github.com/erlang/otp/issues/5255

This combination works (tested on Windows so far):

# terminal 1
.\run-openssl-server.ps1
# terminal 2
.\run-java-tlsclient.ps1

...while this does not:

# terminal 1
.\run-tls-handshake-server.ps1
# terminal 2
.\run-java-tls-client.ps1

Thanks to @acogoluegnes for pointing out his test programs.

IngelaAndin commented 2 years ago

Running the first example I will get 

Given this client's capabilities ([TLSv1.3]), the server prefers protocol=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384

or

Exception in thread "main" java.net.ConnectException: Connection refused (Connection refused)
    at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412)
    at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255)
    at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237)
    at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.base/java.net.Socket.connect(Socket.java:609)
    at net.christopherschultz.ssltest.SSLTest.createSSLSocket(Unknown Source)
    at net.christopherschultz.ssltest.SSLTest.main(Unknown Source)

And in both cases, I will get the alert once.

The second does not work for me:

.\run-openssl-server.ps1
.run-openssl-server.ps1: command not found
lukebakken commented 2 years ago

Hi @IngelaAndin, it's not clear to me what you ran, since I mentioned several commands in this issue. I'm assuming you're using Windows. Could you re-run the commands and copy all of the terminal content, including the command? I will do the same and follow up.

IngelaAndin commented 2 years ago

Oh, I ran everything on linux. Was the last one a windows only?

IngelaAndin commented 2 years ago 
./run-ssltest.sh
[INFO] Connecting to TLS host: localhost, TLS port: 9999
Host [localhost] resolves to address [127.0.0.1]
Testing server localhost:9999
Supported Protocol Cipher
 Rejected  TLSv1.3 TLS_DHE_DSS_WITH_AES_256_GCM_SHA384
 Rejected  TLSv1.3 TLS_DH_anon_WITH_AES_256_CBC_SHA
 Rejected  TLSv1.3 TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA
 Rejected  TLSv1.3 TLS_DH_anon_WITH_AES_256_CBC_SHA256
 Rejected  TLSv1.3 TLS_DHE_DSS_WITH_AES_128_CBC_SHA256
 Rejected  TLSv1.3 TLS_DHE_RSA_WITH_AES_128_CBC_SHA256
 Rejected  TLSv1.3 TLS_ECDHE_RSA_WITH_NULL_SHA
  Timeout  TLSv1.3 TLS_RSA_WITH_AES_256_CBC_SHA
 Rejected  TLSv1.3 TLS_DHE_DSS_WITH_AES_128_GCM_SHA256
 Rejected  TLSv1.3 TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
   Failed  TLSv1.3 SSL_RSA_WITH_DES_CBC_SHA
   Failed  TLSv1.3 SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA
 Rejected  TLSv1.3 TLS_DHE_RSA_WITH_AES_256_GCM_SHA384
   Failed  TLSv1.3 TLS_ECDH_RSA_WITH_NULL_SHA
 Rejected  TLSv1.3 SSL_DH_anon_EXPORT_WITH_RC4_40_MD5
   Failed  TLSv1.3 TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
   Failed  TLSv1.3 TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA
 Rejected  TLSv1.3 TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
   Failed  TLSv1.3 SSL_DH_anon_WITH_DES_CBC_SHA
   Failed  TLSv1.3 TLS_DH_anon_WITH_AES_128_CBC_SHA
 Rejected  TLSv1.3 TLS_RSA_WITH_AES_256_CBC_SHA256
   Failed  TLSv1.3 TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
   Failed  TLSv1.3 TLS_AES_128_GCM_SHA256
 Rejected  TLSv1.3 TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
   Failed  TLSv1.3 TLS_DHE_DSS_WITH_AES_256_CBC_SHA
 Accepted  TLSv1.3 TLS_AES_256_GCM_SHA384
 Rejected  TLSv1.3 TLS_ECDHE_ECDSA_WITH_RC4_128_SHA
 Rejected  TLSv1.3 SSL_DHE_RSA_WITH_DES_CBC_SHA
 Rejected  TLSv1.3 TLS_ECDH_RSA_WITH_AES_128_CBC_SHA
 Rejected  TLSv1.3 TLS_DHE_RSA_WITH_AES_128_CBC_SHA
 Rejected  TLSv1.3 SSL_DH_anon_WITH_RC4_128_MD5
 Rejected  TLSv1.3 TLS_ECDHE_ECDSA_WITH_NULL_SHA
  Timeout  TLSv1.3 TLS_CHACHA20_POLY1305_SHA256
 Rejected  TLSv1.3 TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256
 Rejected  TLSv1.3 SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA
   Failed  TLSv1.3 TLS_RSA_WITH_NULL_SHA256
 Rejected  TLSv1.3 TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA
   Failed  TLSv1.3 SSL_DH_anon_WITH_3DES_EDE_CBC_SHA
   Failed  TLSv1.3 TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384
 Rejected  TLSv1.3 TLS_DHE_RSA_WITH_CHACHA20_POLY1305_SHA256
   Failed  TLSv1.3 TLS_RSA_WITH_AES_128_CBC_SHA256
   Failed  TLSv1.3 TLS_ECDH_anon_WITH_NULL_SHA
 Rejected  TLSv1.3 SSL_RSA_WITH_3DES_EDE_CBC_SHA
   Failed  TLSv1.3 SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA
   Failed  TLSv1.3 TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384
 Rejected  TLSv1.3 TLS_RSA_WITH_AES_128_GCM_SHA256
   Failed  TLSv1.3 TLS_ECDH_anon_WITH_RC4_128_SHA
   Failed  TLSv1.3 SSL_DHE_DSS_WITH_DES_CBC_SHA
 Rejected  TLSv1.3 TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256
 Rejected  TLSv1.3 TLS_RSA_WITH_AES_256_GCM_SHA384
   Failed  TLSv1.3 TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384
 Rejected  TLSv1.3 TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
   Failed  TLSv1.3 SSL_RSA_EXPORT_WITH_DES40_CBC_SHA
   Failed  TLSv1.3 TLS_ECDH_anon_WITH_AES_256_CBC_SHA
 Rejected  TLSv1.3 TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256
   Failed  TLSv1.3 TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
   Failed  TLSv1.3 SSL_RSA_EXPORT_WITH_RC4_40_MD5
 Rejected  TLSv1.3 TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
   Failed  TLSv1.3 TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
   Failed  TLSv1.3 TLS_ECDH_RSA_WITH_AES_256_CBC_SHA
 Rejected  TLSv1.3 TLS_ECDH_anon_WITH_AES_128_CBC_SHA
   Failed  TLSv1.3 TLS_ECDH_ECDSA_WITH_RC4_128_SHA
   Failed  TLSv1.3 TLS_RSA_WITH_AES_128_CBC_SHA
 Rejected  TLSv1.3 TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA
   Failed  TLSv1.3 TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384
   Failed  TLSv1.3 TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA
 Rejected  TLSv1.3 SSL_RSA_WITH_RC4_128_SHA
   Failed  TLSv1.3 TLS_ECDH_ECDSA_WITH_NULL_SHA
   Failed  TLSv1.3 TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256
 Rejected  TLSv1.3 TLS_ECDH_anon_WITH_3DES_EDE_CBC_SHA
   Failed  TLSv1.3 TLS_ECDH_RSA_WITH_RC4_128_SHA
   Failed  TLSv1.3 SSL_DH_anon_EXPORT_WITH_DES40_CBC_SHA
   Failed  TLSv1.3 TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
 Rejected  TLSv1.3 SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA
 Rejected  TLSv1.3 TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384
   Failed  TLSv1.3 SSL_RSA_WITH_NULL_SHA
   Failed  TLSv1.3 TLS_ECDHE_RSA_WITH_RC4_128_SHA
 Rejected  TLSv1.3 TLS_DHE_RSA_WITH_AES_256_CBC_SHA256
   Failed  TLSv1.3 TLS_EMPTY_RENEGOTIATION_INFO_SCSV
   Failed  TLSv1.3 SSL_RSA_WITH_RC4_128_MD5
 Rejected  TLSv1.3 TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
   Failed  TLSv1.3 TLS_DH_anon_WITH_AES_128_CBC_SHA256
   Failed  TLSv1.3 TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
 Rejected  TLSv1.3 TLS_DHE_DSS_WITH_AES_256_CBC_SHA256
   Failed  TLSv1.3 TLS_DHE_RSA_WITH_AES_256_CBC_SHA
   Failed  TLSv1.3 TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
 Rejected  TLSv1.3 SSL_RSA_WITH_NULL_MD5
   Failed  TLSv1.3 TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
   Failed  TLSv1.3 TLS_DH_anon_WITH_AES_128_GCM_SHA256
 Rejected  TLSv1.3 TLS_DHE_DSS_WITH_AES_128_CBC_SHA
   Failed  TLSv1.3 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
   Failed  TLSv1.3 TLS_DH_anon_WITH_AES_256_GCM_SHA384
Given this client's capabilities ([TLSv1.3]), the server prefers protocol=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384

./run-tls-server.sh 
=ERROR REPORT==== 12-Nov-2022::13:52:38.591227 ===
ssl:handshake Error: {error,closed}
=ERROR REPORT==== 12-Nov-2022::13:52:39.555862 ===
ssl:handshake Error: {error,closed}
=ERROR REPORT==== 12-Nov-2022::13:52:39.558272 ===
ssl:handshake Error: {error,closed}
=ERROR REPORT==== 12-Nov-2022::13:52:39.559246 ===
ssl:handshake Error: {error,closed}
=ERROR REPORT==== 12-Nov-2022::13:52:39.560276 ===
ssl:handshake Error: {error,closed}
=ERROR REPORT==== 12-Nov-2022::13:52:39.561357 ===
ssl:handshake Error: {error,closed}
=ERROR REPORT==== 12-Nov-2022::13:52:39.562133 ===
ssl:handshake Error: {error,closed}
=ERROR REPORT==== 12-Nov-2022::13:52:39.651291 ===
ssl:handshake Error: {error,
                         {tls_alert,
                             {handshake_failure,
                                 "TLS server: In state wait_cv at tls_handshake_1_3.erl:2077 generated SERVER ALERT: Fatal - Handshake Failure\n \"CertificateVerify uses unsupported signature algorithm\""}}}
=ERROR REPORT==== 12-Nov-2022::13:52:40.655236 ===
ssl:handshake Error: {error,closed}
=ERROR REPORT==== 12-Nov-2022::13:52:40.657603 ===
ssl:handshake Error: {error,closed}
=ERROR REPORT==== 12-Nov-2022::13:52:40.658567 ===
ssl:handshake Error: {error,closed}
=ERROR REPORT==== 12-Nov-2022::13:52:40.659479 ===
ssl:handshake Error: {error,closed}
=ERROR REPORT==== 12-Nov-2022::13:52:40.660275 ===
ssl:handshake Error: {error,closed}
=ERROR REPORT==== 12-Nov-2022::13:52:40.661234 ===
ssl:handshake Error: {error,closed}
=ERROR REPORT==== 12-Nov-2022::13:52:40.662055 ===
ssl:handshake Error: {error,closed}
=ERROR REPORT==== 12-Nov-2022::13:52:40.662855 ===
ssl:handshake Error: {error,closed}
=ERROR REPORT==== 12-Nov-2022::13:52:40.663671 ===
ssl:handshake Error: {error,closed}
=ERROR REPORT==== 12-Nov-2022::13:52:40.664462 ===
ssl:handshake Error: {error,closed}
=ERROR REPORT==== 12-Nov-2022::13:52:40.665236 ===
ssl:handshake Error: {error,closed}
=ERROR REPORT==== 12-Nov-2022::13:52:40.666010 ===
ssl:handshake Error: {error,closed}
=ERROR REPORT==== 12-Nov-2022::13:52:40.666785 ===
ssl:handshake Error: {error,closed}
=ERROR REPORT==== 12-Nov-2022::13:52:40.667835 ===
ssl:handshake Error: {error,closed}
=ERROR REPORT==== 12-Nov-2022::13:52:40.668780 ===
ssl:handshake Error: {error,closed}
=ERROR REPORT==== 12-Nov-2022::13:52:40.669539 ===
ssl:handshake Error: {error,closed}
=ERROR REPORT==== 12-Nov-2022::13:52:40.670280 ===
ssl:handshake Error: {error,closed}
=ERROR REPORT==== 12-Nov-2022::13:52:40.671102 ===
ssl:handshake Error: {error,closed}
=ERROR REPORT==== 12-Nov-2022::13:52:40.671804 ===
ssl:handshake Error: {error,closed}
=ERROR REPORT==== 12-Nov-2022::13:52:40.694956 ===
ssl:handshake Error: {error,
                         {tls_alert,
                             {handshake_failure,
                                 "TLS server: In state wait_cv at tls_handshake_1_3.erl:2077 generated SERVER ALERT: Fatal - Handshake Failure\n \"CertificateVerify uses unsupported signature algorithm\""}}}

So actually I got alert twice!

IngelaAndin commented 2 years ago 
./run-openssl-server.ps1 
./run-openssl-server.ps1: line 1: syntax error near unexpected token `[string]$TlsHost='*','
./run-openssl-server.ps1: line 1: `param([string]$TlsHost='*', [int]$TlsPort=9999)'
IngelaAndin commented 2 years ago

Actually, your test is testing a lot of TLS-1.2 cipher suites (as well as TLS-1.3 ones) against a TLS-1.3 server so naturally, all TLS-1.2 ones will fail. And the java test program seems to fail due to some close/shutdown race every now and then. Maybe it is because I have ancient java (it happened to be 11.0.17) It would help if you could give me a single test that you think should work but does not.

lukebakken commented 2 years ago

I was just going to start simplifying things! I'll just provide Linux scripts. FYI the scripts ending in .ps1 are for Powershell and I only ran them on Windows.

lukebakken commented 2 years ago

I've simplified the reproduction steps:

IngelaAndin commented 2 years ago

Have not had time to check yet, but I am beginning to suspect that #6228 could be a solution to your problem.

lukebakken commented 2 years ago

I compiled Erlang from master just now, and added the certificate_authorites setting here - https://github.com/lukebakken/erlang-otp-6435/blob/lukebakken/certificate-authorities/src/tls_server.erl#L19

When I run the test programs, this is what I see from the Erlang TLS server:

=ERROR REPORT==== 14-Nov-2022::13:32:21.550279 ===
ssl:handshake Error: {error,
                         {tls_alert,
                             {certificate_required,
                                 "TLS server: In state wait_cert at tls_handshake_1_3.erl:1473 generated SERVER ALERT: Fatal - Certificate required\n certificate_required"}}}

Note that this is the same as above, but NOT the same error as I originally reported (CertificateVerify uses unsupported signature algorithm).

lukebakken commented 2 years ago

I have updated the code in my repository, and I amended the reproduction outcome in [this comment]. Note that the two commands to run are the same.

In summary, here are the three errors I have seen:

lukebakken commented 2 years ago

I realize this issue has gone all over the place. I would be happy to close it and start over if it would reduce confusion!

IngelaAndin commented 1 year ago

No need to make a new issue, we can continue here. I could not run a new reproduction. It seems to have its own version of our handshake module that crashes. But If I use the certificates and start a pure erlang server the java client will succeed. This was running master.

Java client is reporting the following for certificate verify

"CertificateVerify": {
  "signature algorithm": ecdsa_secp384r1_sha384
  "signature": {
    0000: 30 65 02 31 00 95 B7 A0   AD 7A 77 38 B7 1F F6 AC  0e.1.....zw8....
    0010: 36 20 DC 9E 10 20 19 98   59 31 30 9C 05 F9 86 54  6 ... ..Y10....T
    0020: 34 2A AE 3B F1 66 65 E7   93 3C 3F 7E 28 75 D8 59  4*.;.fe..<?.(u.Y
    0030: 97 51 90 08 FA 02 30 45   BF 23 56 88 E0 A3 E4 BE  .Q....0E.#V.....
    0040: EC D0 A1 AE 8D 07 D2 23   E8 20 AA 7D 7A 63 2F 58  .......#. ..zc/X
    0050: 98 C5 C5 70 8F 9E A0 DE   50 94 E7 16 03 46 8D DE  ...p....P....F..
    0060: 8C E0 87 7E 84 72 D0                               .....r.
  }
}
lukebakken commented 1 year ago

Wow, I have quite the silly error in my TLS server code. I named the module the same as one in OTP 😮‍💨

lukebakken commented 1 year ago

@IngelaAndin thanks for your patience! I have changed my code and I promise something useful is being demonstrated!

Erlang TLS server code

TLS 1.3 only

TLS 1.3 / 1.2

TLS 1.2 / 1.3

TLS 1.2 only

IngelaAndin commented 1 year ago

Yes, this is much more interesting, once I added the missing v the instruction ;)

TLS 1.3 / 1.2 and TLS 1.2 only I think are behaving as expected. That is sha (sha1) is not supported in TLS-1.3. There is a disclaimer in the RFC that a server could possibly choose to allow a certificate that uses sha in its signature if there are no other better certificates to use for backward compatibility, however so far we have chosen not to support it. If it becomes a big issue we might have to rethink but we were hoping we do not have to!

TLS 1.3 only should behave as TLS 1.3 / 1.2 I think, I will look into fixing that.

TLS 1.2 / 1.3 is a bug in how the application variable is handled, that is it does not sort the list of versions as will normally happen. I will fix that.

lukebakken commented 1 year ago

once I added the missing v the instruction ;)

Sorry! I edited my instructions.

That is sha (sha1) is not supported in TLS-1.3.

Thank you, that gives me something to go back to with this RabbitMQ user.

lukebakken commented 1 year ago

For what it's worth, I have added s.Close() to the end of the Java TLS code to gracefully close the socket. This fixes the badarg error in the ./run-tls-server.sh tlsv1.3-only case. So the badarg must have to do with the interaction of the TLS state machine and non-graceful termination of the socket.

What's also interesting is that by adding s.Close(), the following cases show the CertificateVerify error:

=NOTICE REPORT==== 17-Nov-2022::07:05:09.908495 ===
TLS server: In state wait_cv at tls_handshake_1_3.erl:2071 generated SERVER ALERT: Fatal - Handshake Failure
 - "CertificateVerify uses unsupported signature algorithm"

Finally, though I know this always isn't the best test, openssl s_server does seem to always succeed:

$ ./run-openssl-server.sh -no_ssl3 -no_tls1 -no_tls1_1
+ openssl s_server -accept localhost:9999 -Verify 8 -verify_return_error -CAfile /home/lbakken/development/lukebakken/erlang-otp-6435/certs/rootCA.pem -cert /home/lbakken/development/lukebakken/erlang-otp-6435/certs/cert.pem -key /home/lbakken/development/lukebakken/erlang-otp-6435/certs/key.pem -no_ssl3 -no_tls1 -no_tls1_1
verify depth is 8, must return a certificate
Using default temp DH parameters
ACCEPT
depth=1 CN = Test Root CA
verify return:1
depth=0 CN = ECC CONTROL EE certificate
verify return:1
...
...
subject=CN = ECC CONTROL EE certificate
issuer=CN = Test Root CA
Shared ciphers:TLS_AES_256_GCM_SHA384:TLS_AES_128_GCM_SHA256:TLS_CHACHA20_POLY1305_SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA
Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA+SHA256:RSA+SHA384:RSA+SHA512:DSA+SHA256:ECDSA+SHA224:RSA+SHA224:DSA+SHA224:ECDSA+SHA1:RSA+SHA1:DSA+SHA1
Shared Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:RSA+SHA224
Peer signing digest: SHA384
Peer signature type: ECDSA
Supported groups: x25519:secp256r1:secp384r1:secp521r1:x448:ffdhe2048:ffdhe3072:ffdhe4096:ffdhe6144:ffdhe8192
Shared groups: x25519:secp256r1:secp384r1:secp521r1:x448:ffdhe2048:ffdhe3072:ffdhe4096:ffdhe6144:ffdhe8192
CIPHER is TLS_AES_256_GCM_SHA384
Secure Renegotiation IS supported
DONE
shutting down SSL
CONNECTION CLOSED
IngelaAndin commented 1 year ago

Hum something is fishy! We will have to investigate further. And actually, I might be wrong about the version order bug it seems the sorting is done in another place of the code, so it looks like it is ok.

IngelaAndin commented 1 year ago

Well, I think I found the problem! Both of your test programs are too quick in closing down to observe what is actually happening.

I added a Thread.sleep(5000); to the java client before closing the socket. As in TLS-1.3, the client connection will actually complete before the server is done with client certificate checks so the alert will be received first after the initial connection is established.

On the erlang side not all log messages will have time to print unless you add a timer:sleep(500) after your log calls. This will also allow some ssl process log messages to be visible.

Now I get consistent behavior.

If TLS-1.3 and TLS-1.2 is available version independent of input order (as expected) 

....
<<< Handshake, CertificateVerify
[{algorithm,ecdsa_sha1},
 {signature,<<48,101,2,48,14,117,67,236,49,221,226,115,37,161,80,59,1,155,
              112,15,149,108,209,126,194,213,218,218,122,236,124,33,196,254,
              161,55,212,17,202,217,252,118,156,113,251,54,239,17,19,176,173,
              204,2,49,0,129,132,115,19,188,161,11,102,15,128,176,117,147,
              112,240,51,30,153,166,181,231,71,162,161,110,226,76,242,233,
              124,81,43,176,115,19,32,153,55,254,131,221,109,180,159,131,158,
              223,131>>}]
writing (24 bytes) TLS 1.2 Record Protocol, application_data
0000 - 17 03 03 00 13 cf 7a 65  42 5a e9 88 62 90 d8 a9    ......zeBZ..b...
0010 - d7 a7 97 dc 14 a3 cb c7                             ........
=NOTICE REPORT==== 18-Nov-2022::11:40:49.160677 ===
TLS server: In state wait_cv at tls_handshake_1_3.erl:2071 generated SERVER ALERT: Fatal - Handshake Failure
 - "CertificateVerify uses unsupported signature algorithm"
=ERROR REPORT==== 18-Nov-2022::11:40:49.161850 ===
{error,{tls_alert,{handshake_failure,"TLS server: In state wait_cv at tls_handshake_1_3.erl:2071 generated SERVER ALERT: Fatal - Handshake Failure\n \"CertificateVerify uses unsupported signature algorithm\""}}}

If TLS-1.2 only


>>> Handshake, Finished
[{verify_data,<<149,251,100,33,198,109,217,52,91,220,217,122>>}]
writing (45 bytes) TLS 1.2 Record Protocol, handshake
0000 - 16 03 03 00 28 ed f3 57  31 05 d5 f7 77 85 5d ee    ....(..W1...w.].
0010 - 64 a3 1a 71 3d 84 0c de  ec 1b eb ea c9 88 76 16    d..q=.........v.
0020 - 23 32 41 0f 35 d8 41 d5  0b 16 83 fb 61             #2A.5.A.....a
=INFO REPORT==== 18-Nov-2022::11:54:20.547026 ===
{ok,{sslsocket,{gen_tcp,#Port<0.5>,tls_connection,
                        [{option_tracker,<0.113.0>},
                         {session_tickets_tracker,disabled},
                         {session_id_tracker,<0.114.0>}]},
               [<0.117.0>,<0.116.0>]}}

For TLS-1.3 only you get a different alert (which I think is a bug, I will look into this) 


TLS server: In state wait_cv at tls_handshake_1_3.erl:609 generated SERVER ALERT: Fatal - Internal Error
 - badarg
=ERROR REPORT==== 18-Nov-2022::11:59:41.207009 ===
{error,{tls_alert,{internal_error,"TLS server: In state wait_cv at tls_handshake_1_3.erl:609 generated SERVER ALERT: Fatal - Internal Error\n badarg"}}}
lukebakken commented 1 year ago

Thank you @IngelaAndin! I will take all of this into account the next time I see a strange TLS behavior.

IngelaAndin commented 1 year ago

This turned out to be interesting. Although I clearly remember a discussion we had about not supporting legacy sha1 certificates in TLS-1.3 it turns out we have actually implemented and even documented it. However, it does not work for the ECDSA as some algorithm name conventions fail the algorithm selection. So all things considered I will fix it so it works in OTP-25 and we will aim to start phasing out the legacy algorithms starting OTP-26. This will probably be a more popular approach anyway as it gives legacy systems more time to adapt.

lukebakken commented 1 year ago

The certs I provided in my test project are sha256 signed, not sha1. Should that make a difference? I will re-try my test applications with #6490

lbakken@shostakovich ~/development/lukebakken/erlang-otp-6435/certs (main =)
$ openssl x509 -text -in cert.pem
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 1667369504387 (0x18436f78e83)
        Signature Algorithm: ecdsa-with-SHA256
lukebakken commented 1 year ago

Good news, PR #6490 appears to fix all the test cases here!

IngelaAndin commented 1 year ago

It is the client that chooses to use the legacy algorithm for its 'certificate verify'-message.

javax.net.ssl|DEBUG|01|main|2022-11-24 10:35:55.462 CET|CertificateVerify.java:1128|Produced client CertificateVerify handshake message (
"CertificateVerify": {
  "signature algorithm": ecdsa_sha1
  "signature": {
    0000: 30 66 02 31 00 C8 D5 6B   9E 65 3E 03 CE 00 1A 21  0f.1...k.e>....!
    0010: 06 D2 F1 5E D8 25 B1 B8   9B 32 10 7F FA 43 EA FB  ...^.%...2...C..
    0020: 58 84 68 D6 98 C1 2A 56   F4 25 21 23 55 70 DD 2B  X.h...*V.%!#Up.+
    0030: 56 AF C0 B2 F6 02 31 00   83 9D D2 0B 27 72 AC 50  V.....1.....'r.P
    0040: C9 2A 04 2F C3 A7 1D A4   98 B7 5C 3E D1 0C 29 2F  .*./......\>..)/
    0050: CA 51 68 EA 42 F2 24 A0   50 D0 BD E2 CA A6 8B 62  .Qh.B.$.P......b
    0060: 46 63 B5 22 65 15 6D 09                            Fc."e.m.
  }
}

The erlang server uses 

javax.net.ssl|DEBUG|01|main|2022-11-24 10:35:55.443 CET|CertificateVerify.java:1163|Consuming CertificateVerify handshake message (
"CertificateVerify": {
  "signature algorithm": ecdsa_secp384r1_sha384
  "signature": {
    0000: 30 66 02 31 00 89 58 B7   28 1E F1 9A 2C 23 F2 E4  0f.1..X.(...,#..
    0010: 99 2B 6B 66 37 09 3B 7E   7F E9 2C E0 44 3E 62 3E  .+kf7.;...,.D>b>
    0020: 2E 0F 3C FE 30 FC 58 DF   98 99 19 E5 2F 89 9A 32  ..<.0.X...../..2
    0030: E9 BA 05 EF 02 02 31 00   BF 85 C6 6E 8C 6C 6A 8C  ......1....n.lj.
    0040: FF BC 08 39 57 3F A4 8B   F5 49 64 95 7A 03 C9 A7  ...9W?...Id.z...
    0050: 46 45 CE EA 64 D8 5F F7   41 D0 4F 89 BE AB 95 31  FE..d._.A.O....1
    0060: 5E 65 3D 61 03 06 B1 61                            ^e=a...a
  }
}
lukebakken commented 1 year ago

Thank you for the explanation, that's interesting!

arunarnim commented 1 year ago

Thank you @IngelaAndin @lukebakken. @IngelaAndin - I can see PR 6490 is in merged state, can you please let me know which official release of Erlang version will this PR fix be available so that I can download that official Erlang release and test this PR fix with my product?

IngelaAndin commented 1 year ago

As OTP-25.2 will be released fairly soon now I will not make a separate patch for this! So OTP-25.2 is the answer.