Does NATS support MQTT?

[hoang17]

I would like to use NATS as MQTT broker. Is it possible?

[derekcollison]

Seems QoS2 has some pieces in the broker but is mainly a 4-way handshake between sender and receiver. Will dig in more and see what we can do as we get into the impl.

[vinayxbox]

Any update here ?

[derekcollison]

I m hoping to have work start in a few weeks.

[sonwa]

Can mqtt be used?

[derekcollison]

It is in the works, but just starting. Hope to deliver early Q3.

[ajaegle]

Is there any design document on how managing authentication for mqtt clients and authorization on topics will be handled? I’m really curious to learn more about the current state of the mqtt integration and would like to support testing some early versions for feedback.

[derekcollison]

The mqtt work is being led by @kozlovic, so he may have some input.

[kozlovic]

@ajaegle For MQTT clients auth, we don't have much choice (we plan to support 3.1.1 initially). It will be either username/password and/or TLS. The mqtt{} block in the configuration will be able to provide its own set of users and/or tls configuration. There is a branch: https://github.com/nats-io/nats-server/tree/mqtt which is being worked on. Note that this branch may be updated and sometimes force-pushed at any time without warning, so if you work from this branch, be aware of that.

Note that at this point there is only QoS 0 and "clean session" support only. The rest will trickle slowly.

[nmarcetic]

This is huge so far. Great progress @kozlovic I didn't expect to see everything from scratch :) But IMHO, Not using paho is a good decision. Did you consider maybe to pull out this MQTT part from nats Core and use it as lib, #later ? It will be hell to maintain very soon. Checking code

[kozlovic]

@nmarcetic Maybe, but I am not sure. As you can see, this is already quite in the guts of the server and that's before any use of persistence which will require accessing JetStream internals. So I am not sure it will be easy to externalize this work in separate library. As for not using an existing library, we try to minimize dependencies and as explained above it may have been difficult anyway. Also, if there is a bug in our implementation it will be easier to fix :-).

[nmarcetic]

@kozlovic I see it will not be easy :) Stil I think you can keep server stuff in separate lib and have just handlers (check out what we exposed here) in nats core where you will handle the logic, but keep this packet parsing, etc (1.5k + LOC)... in separate lib. Anyway, maybe its better to get working integration with nats ASAP, then refactor and polish :) Do you have some kind of rough roadmap somewhere public?

[max-koehler]

@kozlovic great work! can you give any estimation or insights into the roadmap for QoS 1 and retained messages?

[kozlovic]

Late Q4: https://github.com/nats-io/nats-general#nats-roadmap. Note this will be based on MQTT v3.1.1 (in roadmap I see 3.1).

[nqd]

@kozlovic do you have a list of open tasks/issues that we can contribute?

[petedavis]

@kozlovic will the retained messages publish to native nats clients? And will a native nats client be able to publish retained messages, and reuse this capability that has been produced for MQTT producers?

[kozlovic]

@nqd Sorry for the delay and thank you for the offer. However, I enter the phase of retain/persistence and this will be really going deep in the guts of the server and have to mix with JetStream, so not sure what to ask community to contribute at this time. However, I would like ask for users feedback when something is ready for testing.

Also, I would like to know what users are especially looking for and if that matches what is going to be offered. That is, we have plans for MQTT v3.1.1 and only QoS 0 and 1. If everybody says that they need QoS 2 or it is useless, then we have a problem :-)

@petedavis:

will the retained messages publish to native nats clients

To be clear, are you saying that if MQTT publishes a retained message, you would want NATS subscriptions that start to receive this message? Or simply are MQTT published retained messages sent to existing NATS subscriptions? If the later then yes: an MQTT produced message will be delivered to all matching subscriptions (mqtt and NATS). However, having NATS Subscriptions that start receive MQTT retained messages would mean that we need to go through the list of all retained subjects to see if it matches the newly created subscription: this is what needs to be done for MQTT subs, but not sure if this is something we want to add to the non MQTT code path to be honest.

And will a native nats client be able to publish retained messages

Until recently I would have to say no since you would have no way that I can think of (except maybe through config saying that all messages published to given subject from NATS are considered retained) to signal this behavior. With the introduction of NATS headers (slowly rolling into our client libraries), then it would be possible to notify that the NATS message should be retained, but that leads to performance considerations because now it means inspecting headers (which is very costly) for each incoming NATS message.

[nqd]

@kozlovic will test whenever MQTT feature release. I have been using MQTT for a while, no need QoS2 so far.

[nmarcetic]

@kozlovic If everybody says that they need QoS 2 or it is useless, then we have a problem :-) We would ask for QoS 3 but there is no such service, yet :) If you ask the community, we will always ask for good but cheap, rather then bad and expensive. Joking.. but really, we expect the best possible effort, always.

IMHO, QoS 1 is quite sufficient for most use cases + Nats don't support QoS 2.

However, I would like ask for users feedback when something is ready for testing. I would like to help also. I am more interested for contribution like participation in the development when you need help :)

[antong]

QoS 0 and QoS 1 is good! "Exactly once" might sound desirable, but for most practical applications, it just gives a false sense of security. For instance in the SMS sending example given as an example for when QoS 2 would be required: How does the consumer that would receive the exactly-once message from the broker guarantee that exactly one SMS is delivered? I highly doubt whatever device or service used to actually send SMS messages can give the exactly-once guarantees required for this to make a difference.

For one project, I'm currently using mosquitto with authentication and authorization based on mutual TLS (client certificates). For authorization, I have an ACL with patterns that include the MQTT user name, for example ("pattern read client/%u/to/#" and "pattern write client/%u/from/#"). Mosquitto fills in the user name (%u) using the authenticated client certificate subject CN. In this way mosquitto does not need to know anything about any client accounts, as it is all outsourced to an external, trusted PKI. Will something like this be possible with NATS MQTT?

Thank you for working on this!

[kozlovic]

.. I highly doubt whatever device or service used to actually send SMS messages can give the exactly-once guarantees required for this to make a difference.

We definitively agree on that :-)

In this way mosquitto does not need to know anything about any client accounts, as it is all outsourced to an external, trusted PKI. Will something like this be possible with NATS MQTT?

I would need to look into what mosquitto does in more details. As for NATS, we do offer authentication with user/password and/or TLS, including TLS map which allows to extract the user name to use from the certificate. NATS offers ACLs through permissions and accounts isolation. You can read more about auth here: https://docs.nats.io/nats-server/configuration/securing_nats/authorization, and accounts here: https://docs.nats.io/nats-server/configuration/securing_nats/accounts.

[autodidaddict]

Is there any updated ETA for native MQTT support in NATS?

[cubic3d]

Is there any updated ETA for native MQTT support in NATS?

https://github.com/nats-io/nats-server/releases/tag/v2.2.0

[kozlovic]

Sorry for not notifying this issue. Indeed, MQTT is now available with the NATS Server 2.2.0 release. Here is some documentation regarding MQTT support in NATS Server: https://docs.nats.io/nats-server/configuration/mqtt

[sampras343]

@kozlovic Is it possible to connect NATS or NATS-MQTT to another mqtt server? More like a bridge connection between two brokers?

[cbrake]

one approach would be to write a client app that would subscribe to NATs messages and forward things back and forth to the MQTT broker. It seems this functionality would be fairly custom for your application so not sure if it belongs in the nats-server.

[kozlovic]

@sampras343 NATS Server itself does not have a bridge feature, but the MQTT broker you are using may have one (case of Mosquitto for instance). Otherwise, as @cbrake mentioned, using a single sub/pub to do that bridge should work too and may be letting you customize even better what should be bridged.

[sampras343]

@cbrake @kozlovic Thanks for the updates! Much appreciated! 2 follow up questions:

1. Any timeline/ETA for MQTT v5 support?
2. Any possibilities that bridging will be coming up in near future? ETA?

[kozlovic]

There are no plans at the moment for v5 support nor bridging feature. For the bridge, look at your existing broker since they may have already the bridge feature (I think only one side is enough), or at a custom bridge with a sub/pub.

[voycey]

MQTTv5 is a pre-requisite, there are a bajillion services supporting MQTTv3, its an ancient protocol kept going by the fact that no one seems to bother implementing MQTTv5