mosquitto allows cafile but doesn't go into TLS mode, creating protocol errors.

[gbronner]

I was getting a lot of Error: Protocol error issues when attempting to use SSL. I believe that the source of the issue is that mosquitto_pub assumes TLS/SSL when you use the --cafile argument, while mosquitto never calls net__tls_server_ctx(listener) unless the certfile and keyfile arguments are passed.

Details: Version 2.0.18

Mosquitto is started with

allow_anonymous true
cafile <path>/root2.crt
connection_messages true
listener 8883 localhost
log_dest stderr
log_timestamp true
log_timestamp_format %Y-%m-%dT%H:%M:%S
log_type all
socket_domain ipv4
tls_version tlsv1.2

It will start correctly. However, the function net__tls_server_ctxnever gets called, so the ssl_ctx in the listener never gets initialized.

mosquitto_pub -p 8883 --id "asdf" -m "blah" -d -t "asdf" --cafile <path>/root2.crt -h localhost -V mqttv5

It will generate a connect packet with the correct CONNECT_CMD, and then a 2 byte value of '4' for the protocol length. I believe that it will then SSL encode this.

However, on receipt, mosquitto will attempt to decode the connection message without using SSL The test
if(slen != 4 /* MQTT */ && slen != 6 /* MQIsdp */){ will fail (I got a value of 256 or 257 in my tests) and the rest of the packet looks like garbage.

It seems like this configuration should be explicitly banned or handled correctly, as the current situation creates a lot of confusion.

Note that this is related to a prior bug that was marked as done, but not actually done.

[gbronner]

Prior bug was 1836

[ralight]

@gbronner As it mentions on that other issue, you need certfile and keyfile to enable TLS mode for a listener:

listener 8883
certfile <server.pem>
keyfile <server.key>

On the broker side, the cafile option is only needed if client certificates are in use. On the client side, cafile is often needed to verify the server certificate.