search

project-akri / akri-docs

Documentation for Akri
https://docs.akri.sh/
Apache License 2.0
8 stars 21 forks source link

New proposal for MQTT Discovery Handler #80

Open diconico07 opened 1 year ago

diconico07 commented 1 year ago

This is a new proposal for an MQTT Discovery Handler, this proposal tries to keep things as simple as possible while still solving most use cases

This is part of project-akri/akri#106

diconico07 commented 1 year ago

I created a reference implementation of a Discovery Handler for this proposal, you can find it at suse-edge/akri-mqtt-discovery-handler.

abhijith-hr commented 11 months ago

I took a quick look at all previous discussions regarding this topic and the new proposal. Based on my knowledge, as I work with sub-components of cutting and welding machines that use MQTT, the new proposal from @diconico07 is precise to discover the MQTT-supported devices/machines in the shopfloor backed by a common external MQTT broker running on a centralized location, for eg, on a factory server and also to advertise the discovered device resources to Kubernetes based consumer applications.

One thing to note is that the MQTT Discovery Handler doesn't need to run on every node of the cluster. It is only relevant for something like UDEV device discovery. Every MQTT Discovery Handler is a client of the same MQTT broker in the network. It is better to have a cluster-wide Discovery Handler for this use case.

diconico07 commented 11 months ago

One thing to note is that the MQTT Discovery Handler doesn't need to run on every node of the cluster. It is only relevant for something like UDEV device discovery. Every MQTT Discovery Handler is a client of the same MQTT broker in the network. It is better to have a cluster-wide Discovery Handler for this use case.

I do not completely agree on this, while in theory you can, you may not want to, as if you lose connection between a node or set of nodes and the MQTT broker, then having a DH per node ensures the workload will be on the working nodes.

Moreover, the current agent architecture doesn't allow to have remote/centralized discovery handlers.

abhijith-hr commented 11 months ago

I agree with you.

Few more tricky but possible scenarios are,

  1. Two or more devices (same machine models from a manufacturer) publishing using same topic and one of the topic payload contains the differentiator like Device/Model ID.

  2. I have machine here from a manufacturer that doesn't even differentiate two same models but different devices at topic or payload level.

    • In this scenario, the MQTT broker must have "log_dest topic" in its configuration, this way broker logs are published to clients with "$SYS/broker/log/#" topic, and in the connection logs the right device ID/connection ID must be picked as unique indentifier.
kate-goldenring commented 11 months ago

@shantanoo-desai copying you here in case you have any thoughts given you've been thinking about Akri + MQTT for awhile