[Protocol Enhancement] Support for discovery of MQTT based devices in local network

[suresh-lc]

Is your feature request related to a problem? Please describe. Currently only REST based devices can be discovered in Home Edge framework. Hence services provided by those devices are only discoverable. Devices supporting other protocols like MQTT are not discoverable in this.

Describe the solution you'd like Method to discover MQTT based devices by adding support of MQTT (running a broker). This should support

Device discovery Topic subscription/publish

Additional context There are various considerations to be made during this process.

1. Currently Home Edge is Peer-Peer, meaning every device talks to every other device. So in similar regards should MQTT broker be running in all devices to maintain the current topology. So all device publish and subscribe to all other devices.
2. Should we run broker only in one device and all other device publish to it. Say device A runs the broker and device B ad C are client. Need to see how to exchange the capabilities of B and C with each other.
3. In case of single broker, broker selection method and fallback mechanism to be thought of.
4. Home Edge maintains the devices discovered by REST separately. And MQTT would be maintaining it separately. We need to either unify or refer both the dbs.
5. How do we expose services offered by MQTT devices , might be custom topics.
6. Identify MQTT library for adopting, probably 'mosquitto'.

[tiokim]

MQTT support is a very good idea in terms of extending the network topology.

[MoonkiHong]

@suresh-lc Thanks for the head-up. Seems good. What about contributing your idea to implement the MQTT client/broker to our project first? 😁

[NiranjanPatil25]

LF Home Edge – MQTT Plans : following are the 2 aspects , which one are we prioritizing to be done by Dec & which one to take in Y22 1 - LF Home Edge Even sync with Cloud a. Current approach : EdgeX Integrated b. New approach : Sync events with cloud (AWS/Azure/G ) Independent of EdgeX

2- Local MQTT a. Discovery : discover local MQTT devices b. Control : control the MQTT devices locally eg SWITCH ON AC c. Eventing : Local event exchange eg : AC sends its current state event to Broker ,

[tiokim]

LF Home Edge – MQTT Plans : following are the 2 aspects , which one are we prioritizing to be done by Dec & which one to take in Y22 1 - LF Home Edge Even sync with Cloud a. Current approach : EdgeX Integrated b. New approach : Sync events with cloud (AWS/Azure/G ) Independent of EdgeX

2- Local MQTT a. Discovery : discover local MQTT devices b. Control : control the MQTT devices locally eg SWITCH ON AC c. Eventing : Local event exchange eg : AC sends its current state event to Broker ,

It would be great if the cloud synchronization feature without edgex could be done this year. 😄

[suresh-lc]

LF Home Edge – MQTT Plans : following are the 2 aspects , which one are we prioritizing to be done by Dec & which one to take in Y22 1 - LF Home Edge Even sync with Cloud a. Current approach : EdgeX Integrated b. New approach : Sync events with cloud (AWS/Azure/G ) Independent of EdgeX 2- Local MQTT a. Discovery : discover local MQTT devices b. Control : control the MQTT devices locally eg SWITCH ON AC c. Eventing : Local event exchange eg : AC sends its current state event to Broker ,

It would be great if the cloud synchronization feature without edgex could be done this year. 😄

Cloud synchronization can be done with HTTP or MQTT. So here we say by Cloud Synchronization using MQTT method where in the MQTT broker would be running at cloud and edge devices would send data to this MQTT endpoints.

Device meta data and device generated data are two aspects which can by synch.

Currently we provide APIs to store data. So we need to provide API to developer to send data to cloud. Internally first the device details (device twin kind) should also be made in cloud to hold the data might be.

We should also identify one of the cloud vendor to start of with might be as first step.

[MoonkiHong]

@suresh-lc Let us separate the topic into 2.

1. Cloud synchronization through MQTT (Yes, it can be done through HTTP. 😅 But I believe that this is not a point of discussion)
2. Local MQTT discovery/control/and eventing: MQTT broker at cloud and edge.

As @NiranjanPatil25 pointed out, let us use this thread to take about item number 2 (Local MQTT), and let another (item number 1 - cloud synchronization through MQTT, but without EdgeX just in case) reside in an independent thread / issue.

[suresh-lc]

@suresh-lc Let us separate the topic into 2.

1. Cloud synchronization through MQTT (Yes, it can be done through HTTP. 😅 But I believe that this is not a point of discussion)
2. Local MQTT discovery/control/and eventing: MQTT broker at cloud and edge.

As @NiranjanPatil25 pointed out, let us use this thread to take about item number 2 (Local MQTT), and let another (item number 1 - cloud synchronization through MQTT, but independent to EdgeX0 in other issue.

Was just thinking the same . Might be we start using the https://github.com/lf-edge/edge-home-orchestration-go/issues/382

[suresh-lc]

Was checking on the working of MQTT client/broker. Clients publishes topics and whichever clients have subscribed to would receive it. Broker is just a facilitator and nothing more.

1. The broker doesn't permanently store the message. Do we intend to store it permanent, should we synch with the coredata.
2. Clients can subscribe to any message which is not even published. So the topics are independent. ClientID/Name is something unique and if new device uses same name then old device is disconnected.
3. The connection is kept alive in MQTT
4. The client identity is not revealed to other devices unless it is sent in payload or so
5. Should Home Edge just support MQTT or something more like aid for service offloading. These are just some thoughts which came across, listing it down here so that we don't loose track of them.

Will collate more points as I read through more.

[tiokim]

Was checking on the working of MQTT client/broker. Clients publishes topics and whichever clients have subscribed to would receive it. Broker is just a facilitator and nothing more.

1. The broker doesn't permanently store the message. Do we intend to store it permanent, should we synch with the coredata.
2. Clients can subscribe to any message which is not even published. So the topics are independent. ClientID/Name is something unique and if new device uses same name then old device is disconnected.
3. The connection is kept alive in MQTT
4. The client identity is not revealed to other devices unless it is sent in payload or so
5. Should Home Edge just support MQTT or something more like aid for service offloading. These are just some thoughts which came across, listing it down here so that we don't loose track of them.

Will collate more points as I read through more.

@suresh-lc Is this comment written in terms of Local MQTT?

[suresh-lc]

@t25kim The above points have been written specific to Local MQTT only. Wanted to have this issue specific for local mqtt only.

[tiokim]

1. The broker doesn't permanently store the message. Do we intend to store it permanent, should we synch with the coredata.

If the message should be stored, the local DB is also one of the candidates.

[suresh-lc]

Thinking of below high level architectural flow

[tiokim]

@suresh-lc Are the boxes running as a separate container or a separate process? Running MQTT Broker as a separate container(or process) is right. However I think MQTT client should be running in the Edge Orchestration. What do you think?

[suresh-lc]

@t25kim : Running Broker as a separate container is right approach. We need to also think of saving the mqtt data to coredata db. Should this be based on user choice or by default needs to be thought upon.