OpenHAB: message seems to be sent back to client that it came from

[dzavalishin]

I'd like to integrate multiple smart home dash boards and controllers using MQTT broker as a central node.

One of MQTT clients is OpenHAB instanse which must, obviously, both send state changes and receive state changes from other cliens. Hence I need OpenHAB to publish and subscribe with the same topic names:

mqtt-eventbus:commandPublishTopic=/openhab/${item} mqtt-eventbus:commandSubscribeTopic=/openhab/${item}

But when I do so and issue a state change from outside of the OpenHAB, all the system starts to oscillate (send on/off messages over and over again).

It seems then that Mosquitto is sending message back to the client which published that message.

Q1. Is it? How does mosquitto check for loops/storms? Q2. Can it be fixed?

[karlp]

1. Yes, of course, that's how pubsub works, per spec.
2. Not that I'm aware of, there's a "allow duplicates" option, but it's more for overlapping subscriptions.

Most systems use separate topics for commands and notifications, or at the very least different formats, to avoid this, I would have thought openhab would have already solved this issue.

[dzavalishin]

If you build multimaster network with all nodes able to control and listen for same topics, it is impossible to use different topic names for send/recv directions. :(

Additionally, if one will be using different topic names, like

mqtt-eventbus:commandPublishTopic=/openhab/out/${item} mqtt-eventbus:commandSubscribeTopic=/openhab/in/${item}

It will mean that all the other subscribers communicate through the OpenHAB: message from dash board will be published as, for example, /openhab/in/Light5B, come to MQTT broker, come to OpenHAB, return back to MQTT as /openhab/out/Light5B and come to other (and originating!) dashboard.

This situation has a lot of drawbacks:

1. Openhab is being related upon, all the system will fail if OpenHAB fails

2. Message turnaround time is too big, some hundreds of msec. For light control desired timing is < 100ms.

3. All the other nodes will STILL have to do it's own duplicate protection, as they will get their own messages back (node -> broker -> openhab -> broker -> same node) with different topic name.

It seems quite obvious that it is meaningless to send message back to originating subscriber. I see no practical case where subscriber really needs it's own message back.

[dzavalishin]

Actually, the problem is even bigger than I supposed. Loop storm brings the whole Unix box down:

"bash: fork: Cannot allocate memory"

(Came to this state within a minute or so...)

I can't even find out who (and how) ate all the memory, mosquitto or OpenHAB. :(

[karlp]

a) think better about your structures, it's not as hard as you think. b) "there's no need to send back to a subscriber" -> Remember the split of pub and sub. You can pub to something without being a subscriber. You can subscribe to something without being a publisher. You can publish to a topic, and get it back on a subscription, and choose to do things with it yourself in your application. This is not RPC.

[dzavalishin]

There is no possible way to synchronize two nodes with MQTT broker if MQTT broker does not remove duplicates (sends everything back). See, for example, more or less same problem described in https://groups.google.com/forum/#!topic/openhab/OIHgjBG5bxc - even with different send/recv topic names we get feedback loop with two openhabs sending back and forth set of state change messages.

[dzavalishin]

And, as far as I see, fix is really simple, subs.c:99:

        if(!leaf->context->id || (leaf->context->is_bridge && !strcmp(leaf->context->id, source_id))){
            leaf = leaf->next;
            continue;
        }

One just has to remove leaf->context->is_bridge condition and it is done.

Or it can be changed to (leaf->context->is_bridge || config_dont_echo)

[karlp]

Except for the minor detail of that making it, you know, not MQTT compliant anymore. Plenty of people are working with this exactly as designed and specced.

[ralight]

@karlp The bridge options are already non-compliant...

@dzavalishin At the moment your options are to have the clients connect as if they were a bridge. This can be done by setting the most significant bit of the MQTT protocol version in the CONNECT packet. For the future, MQTT v5 allows subscriptions to specify that they do not want to receive their own messages back.

[dzavalishin]

@ralight Thanx for understanding. What you propose is seems to be the most correct solution of the problem. Though instead of kicking OpenHAB guys I'd better had a switch in broker conf. to force some client to a bridge mode. :-) It seem to be faster way. I'll still attempt to ask OpenHUB MQTT maintainers for adding high bit to proto version.

If i'll add some enforce_bridge <cliend_id> conf file option to my fork, will you guys accept it to main repository?

[ralight]

With the advent of MQTT v5 and the "no-local" subscription option, I think this issue can be considered resolved.