dtls.c: Handle DTLS1.3 ClientHello when calculating cookie

[mrdeep1]

Do not calculate the cookie using the Extensions as these are different between DTLS1.2 and DTLS1.3

https://datatracker.ietf.org/doc/html/rfc6347#section-4.2.1

When responding to a HelloVerifyRequest, the client MUST use the same parameter values (version, random, session_id, cipher_suites, compression_method) as it did in the original ClientHello. The server SHOULD use those values to generate its cookie and verify that they are correct upon cookie receipt.

https://www.rfc-editor.org/rfc/rfc9147.html#section-5.3

The ClientHello up to, but not including the Extensions is the same for DTLS1.2 and DTLS1.3

[boaks]

That mainly fails then later in the handshake, or?

[mrdeep1]

The regenerated Cookie for second ClientHello does not match the second ClientHello provided Cookie (generated for the first ClientHello and sent in HelloVerifyRequest) , so a second HelloVerifyRequest is sent.

If the client continued to respond to HelloVerifyRequest with yet another ClientHello, then I guess things may eventually work (unnecessary packet exchanges), but WolfSSL sends back an alert for a duplicated HelloVerifyRequest.

[boaks]

I'm still wondering/confused: I understand your description of the processing of DTLS 1.2 / 1.3 mixed handshakes. But that would mean "DTLS 1.3 breaks the backwards compatibility to 1.2". At least if the 1.3 is the client and 1.2 is the server. Not sure, if that is intended by the TLS working group.

[mrdeep1]

When testing WolfSSL DTLS1.3 against DTL1.2 servers using OpenSSL, mbedtls, or gnutls there are no issues, so I strongly suspect (not checked) they only check the legacy compatible options rather than adding in all of the extensions. Certainly looking at wireshark, the extensions are not the same for the second ClientHello.

[boaks]

I guess, there are two understandings of

"same parameter values (version, random, session_id, cipher_suites, compression_method)"

one is, that the list defines exactly the " parameter values", the other, that it defines an example and the extensions may be added. I guess, the first one is the common one. I wrote a e-mail to one of the RFC9147 authors. Let's see if I get an answer. If not, LGTM.

[mrdeep1]

While unclear what should happen with a HelloVerifyRequest (as opposed to a HelloRetryRequest), https://www.ietf.org/archive/id/draft-ietf-tls-esni-17.html#section-6.1.5 will update the ClientHello for the second ClientHello request

[boaks]

Merged alternative implementation with PR #230.