MQTT 5 uTransport topic structure needs to be revisited for d2c & c2d connections

[devkelley]

This issue was raised in the uprotocol-mqtt slack channel:

"a critical flaw in our D2C and C2D topic definitions that there will be too many topics across the millions of vehicles and hundreds of topics per-vehicle if we events are published to the specific topics following: {client_identifier}/{source.authority_name}/{source.ue_id}/{source.ue_version_major}/{source.resource_id}/{sink.authority_name}/{sink.ue_id}/{sink.ue_version_major}/{sink.resource_id} To simplify the communication (and to drastically reduce the # of topics the broker needed to manage), we used a single D2C and C2D topic that was more like: {client_identifier}/{sink.authority_name}/ and the sink authority was added to published events sent to the vehicles (so we can ensure all events made it to the vehicles)."

Desired outcome: A MQTT topic structure that allows for authorization controls at the broker level, conforms to the segment limitations of cloud brokers, and reduces the number of unique topics for each vehicle.

Please use this issue to discuss use cases and proposals.

[jjj-vtm]

In general, even 100+ mio. of topics, are not directly a problem, especially when you have a distinguishing topic element like the vin early in the topic tree. The topic number might be a problem in a clustered mqtt setup since to my knowledge every node in a cluster needs to know every subscription.

Maybe I get it wrong but looking from a vehicle side I would expect that it subscribes to:

c/+/+/+/+/VIN/{sink.ue_id}/+/+

so basically for the C2D communication the number of subscriptions should be the number of ues in this domain which I guess should not be more than 30 or so. From a cloud side it would just be a single subscription to d/+....

[stevenhartley]

Hi @jjj-vtm, What I recall the concern was not about the number of subscriptions as you pointed out we subscribe to a lot of wildcard sections of the topic, but the amount of individual publish topics. I don't understand the problem so I will ask them to comment on this issue now.

[jjj-vtm]

Hi @stevenhartley, ok what I can think of and this is a real problem is when for example every ue would instantiate a UTransport (MQTT) on its own and subscribe to the wild card topic. This would lead to many connections and many unnecessary messages.

Maybe it is already considered in the uProtocol but a Gateway component should solve this issue. For MQTT one would have an internal MQTT broker where each ue subscribes to the very specific topic it wants to listen too and the Gateway would be an UStreamer (MQTT <--> MQTT) which subscribes to the wildcard topic on the cloud broker an forwards all messages to the internal broker. With this topology one would only need a single MQTT connection to the cloud broker and it also has the benefit that one might think of more sophisticated authorization checks on the gateway.

[stevenhartley]

That is what we have but we are questioning if the topic structure can be the same across the brokers so that the utransport implementations can remain the same.