auth.server.cafile property specified in config.properties is not working

dlowrey commented 3 months ago

Expected behavior

When config.properties contains valid values for:

auth.server.cafile
auth.client.cert
auth.client.key

I should be able to do operations like con, pub, sub without having to specify certificate information.

Actual behavior

When attempting to do a con via the interactive shell (started via mqtt sh), I only get:

No trusted certificate found

My colleagues and I have troubleshooted this all day (on Mac OSX and Windows), and we discovered that if we specify the --ca-cert option (con --ca-cert ...) everything works fine, and the other client certificate values get picked up from the config.properties files correctly.

To Reproduce

Must have a MQTT broker set up for mTLS.

Steps

Specify the below keys with valid values in your config.properties file:

auth.server.cafile
auth.client.cert
auth.client.key

Then start interactive shell

mqtt sh

And try to connect to a broker

con -i myIdentifier

Reproducer code

Details

I think a similar issue was opened previously: https://github.com/hivemq/mqtt-cli/issues/125, and we had some MacOSX users who had no issue with this using 4.13.0, but once they upgraded to 4.29.0 it broke.

Affected MQTT CLI version(s): 4.29.0
Used JVM version: 11, 23

LukasBrand commented 3 months ago

Hi @dlowrey, thank you for using the mqtt-cli!

This behavior is quite strange as both the properties configuration as well as the command line options use the same logic to setup mTLS..

Could you please give more details about the used formats and encryption algorithms used in your certificates and private key as well as your TLS properties file setup?

For reference, here are the supported TLS configurations and setup possibilities: https://hivemq.github.io/mqtt-cli/docs/tls A possible issue could be: In order to use TLS with your default values inside the properties configuration file, simply add -s or --secure.

More details:

The keystore loading uses a natural hierarchy to determine precedence over the security provider possibilities.

A keystore provided as input parameter
A private key and related certificate provided as input parameter
A keystore provided as default option inside the cli configuration file

A private key and related certificate provided as default option inside the cli configuration file

Since #125 we created system tests that include multiple tests for TLS and mTLS. For example: src/systemTest/java/com/hivemq/cli/commands/cli/publish/PublishConnectMTlsST.java

@CartesianTest
@Timeout(value = 3, unit = TimeUnit.MINUTES)
void test_properties_encrypted_pem_private_keys_mutualTls(
        @CartesianTest.Values(chars = {'3', '5'}) final char mqttVersion,
        @CartesianTest.Enum final @NotNull TlsVersion tlsVersion,
        @CartesianTest.Values(strings = {
                "pkcs1.aes256.pem",
                //"pkcs1.camellia256.pem",
                //"pkcs1.des.pem",
                "pkcs1.des3.pem", "pkcs8.aes256.pem",
                //"pkcs8.camellia256.pem",
                //"pkcs8.des.pem",
                "pkcs8.des3.pem"}) final @NotNull String clientKeyType) throws Exception {
    final String certificateAuthorityPublicKey = Resources.getResource("tls/certificateAuthority/ca.pem").getPath();
    final String clientPublicKey = Resources.getResource("tls/client/client-cert.pem").getPath();
    final String clientPrivateKey = Resources.getResource("tls/client/client-key." + clientKeyType).getPath();

    final Map<String, String> properties = Map.of("auth.client.cert",
            clientPublicKey,
            "auth.client.key",
            clientPrivateKey,
            "auth.server.cafile",
            certificateAuthorityPublicKey,
            "auth.client.key.password",
            "clientKeyPassword");

    final List<String> publishCommand = List.of("pub",
            "-h",
            hivemq.getHost(),
            "-p",
            String.valueOf(hivemq.getMqttTlsPort()),
            "-V",
            String.valueOf(mqttVersion),
            "-i",
            "cliTest",
            "-t",
            "test",
            "-m",
            "message",
            "-s",
            "--tls-version",
            tlsVersion.toString(),
            "-d");

    final ExecutionResultAsync executionResult = mqttCli.executeAsync(publishCommand, Map.of(), properties);
    executionResult.awaitStdOut("finish PUBLISH");
    assertConnectPacket(hivemq.getConnectPackets().get(0),
            connectAssertion -> connectAssertion.setMqttVersion(MqttVersionConverter.toExtensionSdkVersion(
                    mqttVersion)));

    assertPublishPacket(hivemq.getPublishPackets().get(0), publishAssertion -> {
        publishAssertion.setTopic("test");
        publishAssertion.setPayload(ByteBuffer.wrap("message".getBytes(StandardCharsets.UTF_8)));
    });
}

LukasBrand commented 3 months ago

https://hivemq.kanbanize.com/ctrl_board/22/cards/23977/details/

LukasBrand commented 2 months ago

I am closing this issue for now. Feel free to reopen anytime if the issue still persists after applying my proposed solution.

hivemq / mqtt-cli