Mutual TLS not working as expected

[facucc]

AWS IoT is not rejecting invalid or unregistered certificates during the TLS handshake. Instead, it ensures certificate validity when the device tries to connect using MQTT or perform operations like publish/subscribe.

Reproducible test case

openssl req -x509 -newkey rsa:2048 -keyout invalid-key.pem -out invalid-cert.pem -days 365 -nodes -subj "/CN=InvalidClient"

openssl s_client -connect A111111-ats.iot.us-east-1.amazonaws.com:8883 \
    -cert invalid-cert.pem \
    -key invalid-key.pem \
    -CAfile AmazonRootCA1.pem -tls1_2

Mutual TLS also works with the same certificate using the example of mutual auth

[INFO] [DEMO] [mqtt_demo_mutual_auth.c:677] Establishing a TLS session to A111111-ats.iot.us-east-1.amazonaws.com:8883.
[ERROR] [Transport_OpenSSL_Sockets] [openssl_posix.c:843] Failed to receive data over network: SSL_read failed: ErrorStatus=EVP lib.
[ERROR] [MQTT] [core_mqtt_serializer.c:2613] A single byte was not read from the transport: transportStatus=-1.
[ERROR] [MQTT] [core_mqtt.c:2424] CONNACK recv failed with status = MQTTRecvFailed.
[ERROR] [MQTT] [core_mqtt.c:2749] MQTT connection failed with status = MQTTRecvFailed.
[ERROR] [DEMO] [mqtt_demo_mutual_auth.c:1168] Connection with MQTT broker failed with status MQTTRecvFailed.

I'm also sharing the Wireshark capture

[cookpate]

AuthN/AuthZ is deferred until after a TLS handshake succeeds. This is the expected behavior from AWS IoT.

TLS verifies that the client certificate matches its keys. After the TLS handshake, and after the first application message is received, AWS IoT will verify that the client certificate is registered on the account.

This allows for JITR, custom authentication rules, etc, to run after TLS confirms that the client certificate and keys match mathematically.

Additionally, x509 certificates without a CA can be registered on AWS IoT.

Is there a use case where a device needs to differentiate AuthN from AuthZ errors? There would be a similar Wireshark trace when connecting with a registered client certificate but use an API for which the certificate allows no authorization.

[facucc]

The purpose of mTLS is to perform bidirectional authentication during the TLS handshake. This means that both the client and the server must exchange and validate their certificates as part of the handshake process. Deferring the client certificate validation until after the handshake defeats the purpose of mTLS, which is to ensure both parties are authenticated before any data exchange occurs. If the server cannot validate the client's certificate, the handshake must fail immediately, preventing the connection from being established.

[aggarg]

As @cookpate explained, this is done intentionally to support Just In-Time Registration (JITR) and other custom authentication rules. The client is still not able to exchange any data with AWS IoT with an invalid certificate. Can you share the usecase/problem that you are trying to address?

[facucc]

Thank you for the clarification. I understand the provisioning mechanisms such as Just-In-Time Registration (JITR) and other custom authentication rules that AWS IoT offers. However, the concept of mutual TLS (mTLS) is being incorrectly applied here.

I’m sorry for the insistence, but in accordance with the TLS protocol, the client certificate validation must occur during the handshake process. If the certificate presented by the client is invalid or unrecognized, the TLS handshake should immediately fail, rejecting the connection.

[aggarg]

What you are suggesting is a change in AWS IoT and would break features like JITR and custom authentication. I am not sure AWS IoT would want to break these features especially when it is not causing any harm. I'll still forward this to the AWS IoT team. I am closing this issue. Feel free to re-open or create a new one if you need any more help.