eclipse-uprotocol / up-spec

uProtocol Specifications
Apache License 2.0
31 stars 25 forks source link

MQTT5 Transport Modes: Device-2-Device vs uE-2-uE #201

Closed stevenhartley closed 1 month ago

stevenhartley commented 1 month ago

Some MQTT brokers (like Azure) have a limits on the amount of subscriptions and segments for the MQTT topic that would make the current proposed specification non-compatible. Azure MQTT Broker only supports a maximum of 8 segments whereas uProtocol defines 9. Furthermore, Azure also limits the number of subscriptions to 50 that will not work with the current uProtocol specifications that require an MQTT_SUBSCRIBE for every remote topic .

The agreed upon solution was to go back to a smaller and simplified topic definition for D2D communication (between streamers) similar to what we currently use in production that will work with broker implementations like Azure that is the topic: /{source.authority_name}/{sink.authority_name}

Devices then (when using the transport in D2D mode) MQTT_SUBSCRIBE to anything sent to them +/{my_authority_name} and then MQTT_PUBLISH (send) to the destination topic based on the UUri in the sink (ex. {my_authority_name}/{cloud_authority_name}

Unfortunately the above simplifications won't work for published events as the sink address is not populated inside of published uMessages that are fed to the transport, that is why I was reminded why we manually inserted the sink address inside of published messages before we sent them to the streamer in 1.3.6 implementations in production.

so in summary, to support the above proposal, we have to change the specifications to not invalidate publish uMessages that contain sink (change in uattributes.adoc) and update mqtt5.adoc to support the 2 modes (D-2-D and uE-2-uE).

sophokles73 commented 1 month ago

It is not really clear to me which use case (scenario) we are talking about in this issue. Can you provide a more detailed scenario description @stevenhartley ?

stevenhartley commented 1 month ago

Use case I'm referring to is when a message is published in say Linux (high-compute) and there is a subscriber in the cloud who wants to receive the event. The streamer will register to receive this event (because it queried the uSubscription service for all remote subscribers). When the streamer gets this event, it calls MQTT5UTransport.send() to redirect it to the cloud. Now the transport gets this published event and there is no sink in the UMessage so it cannot use the new simplified D2D MQTT5 topic definition we've defined. The workaround that we did in production for 1.3.6 of the protocol was to get the streamer to populate the sink address in the message before sending it over MQTT.

We need sink so that we do not have to mqtt5 subscribe to ever published event within the vehicle as that would cause us to run out of subscriptions per the limit discussed above.

sophokles73 commented 1 month ago

Got it. Makes a lot of sense to me ...

PLeVasseur commented 1 month ago

Seems reasonable.

And this behavior will just be baked into a D2D mode we enable when used in a streamer. Nothing else special required.

stevenhartley commented 1 month ago

This ends up being a duplicate of #192 so I'll close this issue and then paste the information in 192