TLS server does not honor TLS signature_algorithms extension

[jurajsomorovsky]

According to the TLS 1.2 spec, the server should reject a connection if the SignatureAndHashAlgorithm extension does not contain appropriate algorithms (https://tools.ietf.org/html/rfc5246#section-7.4.3):

   If the client has offered the "signature_algorithms" extension, the
   signature algorithm and hash algorithm MUST be a pair listed in that
   extension.  Note that there is a possibility for inconsistencies
   here.  For instance, the client might offer DHE_DSS key exchange but
   omit any DSA pairs from its "signature_algorithms" extension.  In
   order to negotiate correctly, the server MUST check any candidate
   cipher suites against the "signature_algorithms" extension before
   selecting them.  This is somewhat inelegant but is a compromise
   designed to minimize changes to the original cipher suite design. 

Botan TLS server does not honor the list of algorithms offered in this extension. If the extension contains, for example, SignatureAndHashAlgorithm ANONYMOUS-SHA512, it is still possible to execute a correct TLS handshake with the TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA cipher suite.

To reproduce this issue, generate an RSA key pair and start the Botan server with:

openssl` genpkey -algorithm RSA -out rsa2048.pem -pkeyopt rsa_keygen_bits:2048
openssl req -key rsa2048.pem -new -x509 -days 365 -out rsa2048.crt
./botan tls_server rsa2048.crt rsa2048.pem --port=4433

Download TLS-Attacker 1.1 (https://github.com/RUB-NDS/TLS-Attacker/releases/tag/v1.1) and start it with the following command:

java -jar TLS-Attacker-1.1.jar client -workflow_input invalid-ssh-algorithm.txt -connect localhost:4433

with the following handshake configuration file: invalid-ssh-algorithm.txt

As you will see, the Botan server ignores the SignatureAndHashAlgorithm extension ( with ANONYMOUS-SHA512) and uses RSA-SHA1 for ServerKeyExchange signing.

According to the specification the server should reject the connection. Please note that this is rather a compatibility issue. Note also that with TLS1.1 and 1.0, the server should behave differently.

See also the wireshark trace, where the ClientHello and ServerKeyExchange message contents are displayed:

[randombit]

When TLS::Signature_Algorithms (tls_extensions.cpp) parses the list, it ignores any values that do not decode to something we understand. This includes not only all signature 'anon' but also MD5 and SHA-224 hashes. Likely in the case that the signature algorithms list appears empty later on, it falls back to the (effectively) MTI SHA-1 which as I recall is required to be the default in the case a client omits the extension.

So you would likely see this issue also by sending a client hello extension with RSA-SHA224 only.

[jurajsomorovsky]

I am not sure whether this is exactly the behaviour I am observing.

For example, if I use the following TLS policy configuration:

signature_hashes=SHA-384 SHA-256
signature_methods=ECDSA RSA DSA

I get the following list of algorithms accepted (i.e., the server responds with a ServerHello message):

Supported signature and hash algorithms: RSA-SHA224 DSA-NONE RSA-SHA384 DSA-SHA384 ECDSA-MD5 DSA-SHA224 RSA-SHA256 ANONYMOUS-SHA512 DSA-SHA512 ECDSA-SHA224 ANONYMOUS-NONE ANONYMOUS-SHA384 DSA-MD5 RSA-MD5 ANONYMOUS-SHA224 RSA-SHA1 ECDSA-SHA1 RSA-SHA512 ANONYMOUS-MD5 ANONYMOUS-SHA256 ANONYMOUS-SHA1 ECDSA-SHA512 DSA-SHA256 ECDSA-NONE RSA-NONE DSA-SHA1 ECDSA-SHA256 ECDSA-SHA384

Note that there is also RSA-SHA512 (when using this algorithm, Botan falls back to RSA-SHA1). ECDSA algorithms should also be rejected, because I configured the server with self-signed RSA certificates...I think then it can accept only RSA signature algorithms (the TLS standard is not that clear about this).

Also note that the strict validation behaviour only applies to TLS 1.2 (https://tools.ietf.org/html/rfc5246#section-7.4.1.4.1):

   Note: this extension is not meaningful for TLS versions prior to 1.2.
   Clients MUST NOT offer it if they are offering prior versions.
   However, even if clients do offer it, the rules specified in [TLSEXT]
   require servers to ignore extensions they do not understand.

[securitykernel]

As you will see, the Botan server ignores the SignatureAndHashAlgorithm extension ( with ANONYMOUS-SHA512) and uses RSA-SHA1 for ServerKeyExchange signing.

As far as I understand RFC 5246, ANON shall not be sent in SignatureAndHashAlgorithm:

signature This field indicates the signature algorithm that may be used. The values indicate anonymous signatures, RSASSA-PKCS1-v1_5 [PKCS1] and DSA [DSS], and ECDSA [ECDSA], respectively. The "anonymous" value is meaningless in this context but used in Section 7.4.3. It MUST NOT appear in this extension.

Apart from that, I think @jurajsomorovsky is right. In TLS_Servers choose_ciphersuite(), there must be an additional check that the ciphersuite candidate's signature algorithm is contained in the list of SignatureAndHashAlgorithm.