search

node-red-contrib-ibm-cloud / community

A central repository for co-ordinating the node-red-contrib-ibm-cloud community
Apache License 2.0
3 stars 1 forks source link

Proposal: Messaging nodes #2

Open binnes opened 4 years ago

binnes commented 4 years ago

Messaging systems often provide the backbone to event driven architectures, so we should support the IBM messaging services in Node-RED, as Node-RED works particularly well in event driven scenarios.

MQ and Event Streams are the 2 primary messaging services in the Integration category of the IBM Cloud Catalog.

There is little support for either IBM service in Node-RED currently. However, as Event Streams is built on the Open Source Kafka application, any of the 13 available Kafka nodes should be able to be used with the Event Streams service (though authentication, etc... has to be configured manually).

The MQ service is less well supported with no active nodes available, though the IBM MQ service can support the MQTT and AMQP protocols and there are nodes for those protocols available.

Items to resolve:

Note: The IBM IoT Platform also supports MQTT, but is not included in this proposal

knolleary commented 4 years ago

I know that @dalelane has played a bit with the kafka side of things in Node-RED. Would be good to get his input here.

johnwalicki commented 4 years ago

We should not forget to mention the MQTT based messaging Node-RED nodes created in the past by the Watson IoT engineering teams.

Many of those need a refresh.

binnes commented 4 years ago

We should not forget to mention the MQTT based messaging Node-RED nodes created in the past by the Watson IoT engineering teams.

  • node-red-contrib-ibm-watson-iot
  • node-red-contrib-scx-ibmiotapp
  • node-red-contrib-iot-virtual-device
  • node-red-contrib-ibm-wiotp-device-ops

Many of those need a refresh.

I was keeping the IBM IoT nodes separate from the IBM messaging services (those in the integration category in the cloud catalog). If we want to support the IBM IoT Platform over the open source MQTT protocol then we should raise another proposal for those nodes?

johnwalicki commented 4 years ago

Agree that the IBM IoT nodes can be a separate GitHub Issue, just wanted to mention it here in the context of Messaging services.

dalelane commented 4 years ago

There is little support for either IBM service in Node-RED currently. However, as Event Streams is built on the Open Source Kafka application, any of the 13 available Kafka nodes should be able to be used with the Event Streams service (though authentication, etc... has to be configured manually).

The challenge with this is that Event Streams has always required authentication for Kafka clients, and some of the Node-RED Kafka nodes haven't supported authentication.

I added auth support to one of the more popular nodes for just this reason in https://github.com/hjespers/node-red-contrib-rdkafka/pull/14

With the OpenShift flavour of Event Streams, the new version 10.0.0 released this month allows administrators to disable authentication - that should make it compatible with almost any Kafka nodes (I'm speculating to be honest, I've not tested all of them!) - so things are a little easier there now. But for the IBM Cloud Event Streams service, that's obviously always going to require auth.

dalelane commented 4 years ago

And to add a more positive note, for Node-RED nodes that have a way to provide auth credentials, working with Event Streams becomes super easy! https://dalelane.co.uk/blog/?p=3732

chughts commented 4 years ago

MQ over REST nodes do exist - https://flows.nodered.org/node/node-red-contrib-mqrest-client GET / PUT is supported. PUBLISH and Admin REST capabilities need to be added.