search

peterprib / node-red-contrib-kafka-manager

Implement
GNU General Public License v3.0
22 stars 12 forks source link

Invalid topic if it contains "/" #28

Closed janainascal closed 3 years ago

janainascal commented 3 years ago

Hi there,

I'm having an error when the topic contains "/": Error log

Since it is a common used used pattern in MQTT, and Kafka is commonly used to handle MQTT messages, I think it shouldn't be a issue. In my case I need the "/" so I keep coherence with MQTT messages from others sources.

I hope this contribution is helpful =)

peterprib commented 3 years ago

My code just passes thru to node-kafka and don't believe it does checking. The error appears to be returned from kafka and from my reading it is a know limitation of kafka . See https://www.ibm.com/support/knowledgecenter/SSMKHH_10.0.0/com.ibm.etools.mft.doc/bz91041_.html Understand your issue as MQ uses this character in it topic handling routing.

peterprib commented 3 years ago

If it helps have written another set of nodes that emulate mq topic pub/sub model https://github.com/peterprib/node-red-contrib-pubsubhub. I could add a feature to it to have subscription convert message to topic used for consumption. Then for kafka could add option to translate topics from "/" to say "_-" and reverse however couldn't deal with lost part of topic in reverse translation. The message could then be feed into Kafka. Both these combined would then emulate MQ.

janainascal commented 3 years ago

Hey Peter,

Thanks for the ideia. I'll try it.

peterprib commented 3 years ago

If helps as I was adding wildcard consumer thought I would add feature option on producer to convert "/" to "." as it was easy to do and may help many people.