erlang / otp

Erlang/OTP
http://erlang.org
Apache License 2.0
11.19k stars 2.93k forks source link

Curve secp521r1 not matched on ssl_handshake:curve_to_atom() because of a trimmed leading zero byte #8552

Open maszlasz opened 1 month ago

maszlasz commented 1 month ago

Describe the bug ssl_handshake:curve_to_atom() expects the given curve to be an exact match, even though in my understanding the leading zero bytes in its parameters can be trimmed due to DER encoding, which can happen with secp521r1 and its B parameter. It can be

<<81,149,62,185,97,142,28,154,31,146,154,33,160,182,133,
  64,238,162,218,114,91,153,179,21,243,184,180,137,145,142,241,
  9,225,86,25,57,81,236,126,147,123,22,82,192,189,59,177,
  191,7,53,115,223,136,61,44,52,241,239,69,31,212,107,80,
  63,0>>

which misses the leading zero found here: https://github.com/erlang/otp/blob/72b5b5768deec9c738a97d5b49a6092db7f95ddd/lib/ssl/src/ssl_handshake.erl#L1881-L1905

To Reproduce A private key and a signing request is created with OpenSSL 1.0.2 via: openssl req -new -newkey ec -pkeyopt "ec_paramgen_curve:secp521r1" -keyout example.key -out example.csr [...] which is then signed with digest sha512 and provided to an Erlang server. The server is configured for TLS 1.3 and has the support for ecdsa_secp521r1_sha512. When a client tries to establish a TLS 1.3 connection (with the indication of support for ecdsa_secp521r1_sha512), the server on the reception of the Client Hello tries to decode its own cert and, due to the above issue, has the elliptic curve marked as unsupported. The handshake then fails as the server now has no suitable signature algorithm to select.

The same situation works with curves prime256v1/secp256r1 and secp384r1 (and the matching sign algs), as their curve parameters have no leading zero bytes and they have a correct match on clauses of curve_to_atom().

Expected behavior ssl:curve_to_atom() matching curves with or without leading zeros.

Affected versions Tested on OTP 25.3.2.12.

Additional context Not entirely sure if it's a bug or perhaps I'm missing something here...

IngelaAndin commented 1 month ago

We will look into it, it is vacation period right now so please have some patience.