MQTT with TLS does not connect to my Mosquitto server

[Vayatoalla]

I have a mosquitto server configured with TLS. It allows connections with certificate and with user/password, and I am connecting to it from several type of devices using both options. But when I try to connect from my homeautio docker it fails. I have a "Socket error on client , disconnecting." log in my mosquitto server, and an ERR log in the container: [15:25:48 ERR] Error while connecting with server. [15:25:48 INF] Disconnected. [15:25:48 INF] MQTT Connection closed unexpectedly, reconnecting... [15:25:48 WRN] MQTT Connection failed, retrying...

I have open a new port in my mosquitto server without TLS. This way mqtt woks fine and I'll go on with this option. But anyway, it's a pity that I could not solve the problem with the TLS.

[i8beef]

Did you enable "brokerUseTls" in the "mqtt" section of the appsettings.config?

[Vayatoalla]

Yes: "brokerUseTls": true

[i8beef]

This was a feature added by PR in the HomeAutio.Mqtt.Core project, and honestly I haven't tested it as I don't use TLS for my local implementation. I hadn't heard back from the original author as to whether it worked out for him, but I'm guessing the answer is "no".

Forgive me as I'm just acquainting myself with the MQTT TLS model.

One difference I see off the bat is that it looks like TLS MQTT configs are supposed to use port 8883 instead of 1883. Did you change that port accordingly in your config?

It also looks like there is a whole set of configuration options for the upstream MQTTNet library for TLS, a lot of which deal with certificate chain / certificate authority validation. If you're using a self-signed cert, it looks like I'd have to expose pretty much all of those settings for configuration. If you are using a real cert, I'm not even sure it would pass validation without passing in the CA cert for the signing authority.

[i8beef]

I just pushed another version, and updated some documentation (https://github.com/i8beef/HomeAutio.Mqtt.GoogleHome/wiki/Config:-appsettings.Production.json#mqtt-settings)

Im shooting in the dark a little bit here, but this should allow you to send through all of the configuration options the underlying MQTTNet library would allow, including adding CA cert for verification / auth certificates. I'm interested to see if that gets you working, or any sort of better error.

I would think you need to 1) ensure port, 2) possibly set the allowUntrustedCertificates to true if you're using a self-signed cert on the MQTT broker side, and 3) if, and I think ONLY if, you're using cert based auth, add the cert settings for the client side cert to the certificates array.

[Vayatoalla]

yes: I have changed the port (8883). I am using a self-signed certs. I only want to validate the server certificate. I am trying to connect with user/password, but could try the authentication with a certficate (I have clients with both types of authentication in my mosquitto server).

I have tried the new version, but it fails starting the mqtt. With this log:

[17:07:05 INF] Loaded with configuration from: appsettings.json, /app/config/appsettings.Production.json [17:07:07 INF] User profile is available. Using '/root/.aspnet/DataProtection-Keys' as key repository; keys will not be encrypted at rest. [17:07:08 INF] Starting IdentityServer4 version 2.3.2.0 [17:07:08 INF] Using the default authentication scheme Cookies for IdentityServer [17:07:08 INF] Device configuration file config/googleDevices.json not found, starting with empty set [17:07:08 INF] Service start initiated [17:07:08 FTL] An error occurred starting the application System.AggregateException: One or more errors occurred. (Value cannot be null. Parameter name: BrokerTlsSettings) ---> System.ArgumentNullException: Value cannot be null. Parameter name: BrokerTlsSettings at HomeAutio.Mqtt.Core.ServiceBase.StartAsync(CancellationToken cancellationToken) at Microsoft.AspNetCore.Hosting.Internal.HostedServiceExecutor.ExecuteAsync(Func 2 callback) --- End of inner exception stack trace --- at Microsoft.AspNetCore.Hosting.Internal.HostedServiceExecutor.ExecuteAsync(Func 2 callback) at Microsoft.AspNetCore.Hosting.Internal.HostedServiceExecutor.StartAsync(CancellationToken token) ---> (Inner Exception #0) System.ArgumentNullException: Value cannot be null. Parameter name: BrokerTlsSettings at HomeAutio.Mqtt.Core.ServiceBase.StartAsync(CancellationToken cancellationToken) at Microsoft.AspNetCore.Hosting.Internal.HostedServiceExecutor.ExecuteAsync(Func 2 callback)<--- Hosting environment: Production Content root path: /app Now listening on: http://[::]:5000 Application started. Press Ctrl+C to shut down.

This is my mqtt section in appsettings. I have tried the configuration without any certificate (allowing untrusted certificates), or adding a "passPhrase" in the certificates object. I have also tried changing the allowUntrustedCertificates, ignoreCertificatexxx to false but the error remains.

"mqtt": { "brokerIp": "xxxxxxx.com", "brokerPort": 8883, "brokerUsername": "xxxxxx", "brokerPassword": "xxxxxxx", "brokerUseTls": true, "brokerTlsSettings": { "allowUntrustedCertificates": true, "ignoreCertificateChainErrors": true, "ignoreCertificateRevocationErrors": true, "protocol": "1.2", "certificates": [ { "file": "config/ca.crt" } ] } },

By the way: I am not sure about the use of the certificate array in the configuration, since I understand that it should be necessary to detail the purpose of each certificate (to authenticate the server or the client).

Anyway, I understand that it will not be easy to manage the TLS with the mqttnet library. I have crawled his github and did not find any clear instructions on how to configure the certificates or the format they should have. Fortunately I can continue with MQTT without TLS.

[i8beef]

Doh! Helps if I actually set that right. Try the version that is going up right now.

I dug through his source code a bit, and I don't think there is actually a way to inject a new CA for certificate verification. That certificates array should actually be used for setting up the cert chain for certificate based auth (if you wanted to use it). I think you can leave it null for bare TLS with username/pass auth. The rest of the config I think looks fine though. You'd still get encryption of the connection, but you wouldn't get actual cert validation.

To do that, I think there would have to be two possibilities:

1. I allow you to inject a cert that I use to manually validate the SslStream behind the scene. I really don't want to do that.
2. We figure out how you inject a private CA cert into the docker image in the right place for .NET Core to pick it up and use it during its built in SSL checks.

Of the two, I favor the second as it requires the least amount of special handling, works within the confines of the default .NET Core setup, and where you store CA certs is special per distro anyway. I would think you could install the self-signed root CA cert into your host OS list of CA certs, and then mount that into the docker image at the right place... Not sure if this would work -v /etc/ssl/certs:/etc/ssl/certs:ro.

[Vayatoalla]

:-) Great!. Now it is working with TLS. User auth with user/password. It runs:

1. Without verifying the server CA (setting to false the certificate checks in the MQTT section)
2. Verifying the server CA (setting to true the certificate checks in the MQTT section). I have checked that in this case the MQTT connection will fail unless the container have access to the CA cert.

I have configured my server CA following your instructions and it works like a charm:

1. Copy the private server-ca.crt in my host and update the certificates (update-ca-certificates in Debian)
2. Add "-v /etc/ssl/certs:/etc/ssl/certs:ro" to the "docker run" command.

I didn't try the client authentication with certificate.

[i8beef]

Oh wow, I wasn't actually expecting that cert mount to work. Neat.

[Vayatoalla]

Nice. I close this issue.