Custom Transport Adapter (MQTT)

[hongkongkiwi]

Is there a format or specification available for writing a custom transport adapter?

I'm really interested to use feathersjs with MQTT (for IOT devices), but I couldn't too easily understand what my adapter had to overwrite or support.

Is looking at the socket.io adapter the only way to understand, or is there some docs or info somewhere I could look at?

I like the idea of using feathersjs with any kind of transport method that makes sense (http, websockets, mqtt). Then for IOT devices where bandwidth really matters, I can use mqtt, while for normal real-time stuff I can use websockets.

[daffl]

Indeed, an MQTT transport has been on the radar for a while. Many parts that would be relevant for creating a transport are documented:

• Configure functions to create a plugin
• app.mixins to register functionality when a new service gets added
• Overriding app.setup to start the server you need
• app.services to get a list of all available services

Those are basically the main hook points the existing transports are using.

The main challenge with MQTT and similar messaging protocols is that afaik they don't support sending back acknowledgement data (MQTT can request acknowledgements but it looks like it just returns some flags like the QOS). Acknowledgement data are important to make normal method calls possible. For example with all websocket libraries we can do something like this:

socket.emit('find', 'messages', function(error, data) {
});

The approach for message services that lack this built-in functionality is usually to link incoming messages and their responses through a unique id that is sent with the request. It would be nice to find a general solution for this (part of the reason why I renamed the feathers-socket-commons repository to @feathersjs/transport-commons).

It would be great if I could get some help from someone who has worked with MQTT (also RabbitMQ and ZeroMQ) a little more to start prototyping some of this. Basically what Feathers needs is a way to

• Make "requests" and get back a response
• Identify a single persistent client connection and send targeted messages
• Add authentication information (probably the JWT) to that connection

[binarytracer]

The main challenge with MQTT and similar messaging protocols is that afaik they don't support sending back acknowledgement data (MQTT can request acknowledgements but it looks like it just returns some flags like the QOS). Acknowledgement data are important to make normal method calls possible. For example with all websocket libraries we can do something like this:

this can be done my rabbitMQ with RPC. nodejs + rabbitmq rpc

im currently "trying" to make this happen using rabbitMQ or maybe with NATS.

[claustres]

Worked with AMQP and MQTT a few years ago, mainly to dispatch events from IoT devices or in control centers. In this specific use case we did not use QoS but as far as I know it does exist in specifications, e.g. in https://github.com/mqttjs/MQTT.js#mqttclientpublishtopic-message-options-callback a callback is triggered when the packet is acknowledged and in http://www.squaremobius.net/amqp.node/channel_api.html#channel_ack you acknowledge a message once received. Some dedicated extensions do also exist for this like https://www.squaremobius.net/amqp.node/channel_api.html#confirmchannel.

Not sure what you are looking for in Feathers for the sample you shown with socket.io: ensure that the packet has been received at the transport level or that the server has processed the request ?

[daffl]

What I meant is that the acknowledgement has to be able to actually send back custom data not just an ok status. Two separate channels linked by a unique identifier (UUID) in their payloads (like the correlation id in @binarytracer's RabbitMQ example) is the other option.

[claustres]

In that case I think you need to handle it at framework level since MQTT or AMQP are mainly lower transport protocol not RPC. Something with the correlationId is of course easily possible https://github.com/wolfeidau/mqtt-rpc, some good discussion here as well http://www.hydrogen18.com/blog/rpc-over-amqp.html & https://facundoolano.wordpress.com/2016/06/26/real-world-rpc-with-rabbitmq-and-node-js/.

[PhilWhitehurst]

I am looking at a feathers setup where

1. Client connects through websockets or http
2. The request is either serviced by the local feathers server service or published into a Nats msg queue
3. Remote feathers services subscribe to Nats msg queues and subjects they are interested in
4. Remore feathers services send their response back through the Nats mag queue
5. The response is sent back to the original client and appears as a normal service response with events etc.

I could not see an easy way to doing this as a transport . So have a Nats service define as an application default service with the standard methods. These methods then publish the request into nats etc. I then have a nats subscribe wrapper (with options you can override) you can deploy in the remote feathers services so they can receive the requests through Nats and respond to original client.

Is there an easier more elegant way through feathers transports that I am missing?

[pskfry]

The approach for message services that lack this built-in functionality is usually to link incoming messages and their responses through a unique id that is sent with the request. It would be nice to find a general solution for this (part of the reason why I renamed the feathers-socket-commons repository to @feathersjs/transport-commons).

It would be great if I could get some help from someone who has worked with MQTT

this can be done with MQTT v5 using the responseTopic option included in the message payload. is anyone still working on this?

[scottpageindysoft]

I have a need for this, and as a former control system engineer using MQTT in dozens of projects, I'll be happy to take this on.

Bear in mind, I have a full-time job that takes priority, but since this will only benefit that position it will most likely see a lot of activity.

Can I get suggestions on what to name it? I'd like to stick with what the community uses WRT FeathersJS.

Aside, even without MQTT v5, the previous suggestions about using another channel for responses is the same that I have. Should work without issue.

I'll get started soon.

[daffl]

I'm doing some v5 rework at the moment so this would be perfect timing since I'd also like to look at a native websocket transport which I believe has similar requirements. I think the first step would be to set up a repo that has a basic request/response test for MQTT and then we can go from there.

[scottpageindysoft]

Sounds great! I noticed, yesterday, that Dove (v5) is in the works. I've started this repo to start dev/testing ideas.

feathers-transport-mqtt

[devops724]

Hi There any progress on MQTT Transport Adapter? i have a project with some IOT devices, i want both the admin panel dashboard(websocket client connection) and the devices connected (MQTT Protocol) to feathersjs server so I can show the device's status in the admin dashboard and change the device's status and send the command to devices all in feathersjs server with not any other dependencies how should i implement this , is there such capability in feathersjs , should i setup standalone MQTT server and use message passing between feathersjs and MQTT server?