search

eclipse-leda / leda-utils

Convenience scripts for Eclipse Leda quickstart tutorials
https://eclipse-leda.github.io/leda/
Apache License 2.0
5 stars 5 forks source link

KAD to auto-disable itself when the VUM takes over to avoid state conflicts #43

Closed vasilvas99 closed 1 year ago

vasilvas99 commented 1 year ago

Issue

KAD conflicts (overlaps) with VUM/CUA (Kanto update manager). We would like KAD to auto-disable itself when the container-update-agent (CUA) starts publishing to the containersupdate/desiredstatefeedback topic that it's identifying what actions it should take. Ideally, once disabled the KAD service will stay disabled after reboots/updates as the UM is fully managing the device.

Solution

We add a new module mqtt_listener to KAD under a compile-time feature flag called "mqtt". This module starts a lightweight MQTT client in a separate thread that subscribes to the MQTT broker and listens on the containersupdate/desiredstatefeedback topic for messages from (UM) with status "IDENTIFIED".

Additionally the existence of a lock-file (default path: /var/lib/kanto-auto-deployer/KAD.enabled) is used as persistent marker that KAD has previously seen this message.

When the message with status "IDENTIFIED" is received:

The systemd unit for KAD (example here) is expanded with:

[Unit]
...
ConditionPathExists=/var/lib/kanto-auto-deployer/KAD.enabled

Now systemd will always skip the KAD unit on subsequent restarts if the lock-file does not exist.

On sysVinit systems this can be checked in the runlevel-script in a similar fashion.

Testing:

1) Build QEMUx86-64 from kad-mqtt-poc of https://github.com/SoftwareDefinedVehicle/leda-distro-fork

2) Run image

3) Create a mock_desiredstatefeedback.json message (from UM tests):

{
    "activityId": "123e4567-e89b-12d3-a456-426614174000",
    "timestamp": 123456789,
    "payload": {
        "status": "INDENTIFIED",
        "message": "This is a mock desired state feedback message",
        "actions": []
    }
}

4) mosquitto_pub -t containersupdate/desiredstatefeedback -f mock_desiredstatefeedback.json

5) Check logs: journalctl -u kanto-auto-deployer. Expected:

root@qemux86-64:~# journalctl -u kanto-auto-deployer
Jun 28 11:38:30 qemux86-64 systemd[1]: Started Kanto Auto Deployer.
Jun 28 11:38:30 qemux86-64 kanto-auto-deployer[500]: [2023-06-28T11:38:30Z INFO  kanto_auto_deployer] Running initial deployment of "/data/var/containers/manifests"
Jun 28 11:38:30 qemux86-64 kanto-auto-deployer[500]: [2023-06-28T11:38:30Z INFO  kanto_auto_deployer] Reading manifests from [/data/var/containers/manifests]
Jun 28 11:38:30 qemux86-64 kanto-auto-deployer[500]: [2023-06-28T11:38:30Z ERROR kanto_auto_deployer] An error occurred when connecting to socket: Os { code: 2, kind: NotFound, message: "No such file or directory" }
Jun 28 11:38:30 qemux86-64 kanto-auto-deployer[500]: [2023-06-28T11:38:30Z ERROR kanto_auto_deployer] An error occurred when connecting to socket: Os { code: 2, kind: NotFound, message: "No such file or directory" }
Jun 28 11:38:30 qemux86-64 kanto-auto-deployer[500]: [2023-06-28T11:38:30Z ERROR kanto_auto_deployer] An error occurred when connecting to socket: Os { code: 2, kind: NotFound, message: "No such file or directory" }
Jun 28 11:38:30 qemux86-64 kanto-auto-deployer[500]: [2023-06-28T11:38:30Z ERROR kanto_auto_deployer] An error occurred when connecting to socket: Os { code: 2, kind: NotFound, message: "No such file or directory" }
Jun 28 11:38:31 qemux86-64 kanto-auto-deployer[500]: [2023-06-28T11:38:31Z ERROR kanto_auto_deployer] An error occurred when connecting to socket: Os { code: 2, kind: NotFound, message: "No such file or directory" }
Jun 28 11:38:31 qemux86-64 kanto-auto-deployer[500]: [2023-06-28T11:38:31Z ERROR kanto_auto_deployer] An error occurred when connecting to socket: Os { code: 2, kind: NotFound, message: "No such file or directory" }
Jun 28 11:38:32 qemux86-64 kanto-auto-deployer[500]: [2023-06-28T11:38:32Z ERROR kanto_auto_deployer] An error occurred when connecting to socket: Os { code: 2, kind: NotFound, message: "No such file or directory" }
Jun 28 11:38:33 qemux86-64 kanto-auto-deployer[500]: [2023-06-28T11:38:33Z WARN  kanto_auto_deployer::manifest_parser] Failed to load manifest directly. Will attempt auto-conversion from init-dir format.
Jun 28 11:38:33 qemux86-64 kanto-auto-deployer[500]: [2023-06-28T11:38:33Z INFO  kanto_auto_deployer] Already exists [cloudconnector]
Jun 28 11:38:33 qemux86-64 kanto-auto-deployer[500]: [2023-06-28T11:38:33Z WARN  kanto_auto_deployer::manifest_parser] Failed to load manifest directly. Will attempt auto-conversion from init-dir format.
Jun 28 11:38:33 qemux86-64 kanto-auto-deployer[500]: [2023-06-28T11:38:33Z INFO  kanto_auto_deployer] Already exists [databroker]
Jun 28 11:38:33 qemux86-64 kanto-auto-deployer[500]: [2023-06-28T11:38:33Z WARN  kanto_auto_deployer::manifest_parser] Failed to load manifest directly. Will attempt auto-conversion from init-dir format.
Jun 28 11:38:33 qemux86-64 kanto-auto-deployer[500]: [2023-06-28T11:38:33Z INFO  kanto_auto_deployer] Already exists [feedercan]
Jun 28 11:38:33 qemux86-64 kanto-auto-deployer[500]: [2023-06-28T11:38:33Z WARN  kanto_auto_deployer::manifest_parser] Failed to load manifest directly. Will attempt auto-conversion from init-dir format.
Jun 28 11:38:33 qemux86-64 kanto-auto-deployer[500]: [2023-06-28T11:38:33Z INFO  kanto_auto_deployer] Already exists [hvacservice-example]
Jun 28 11:38:33 qemux86-64 kanto-auto-deployer[500]: [2023-06-28T11:38:33Z WARN  kanto_auto_deployer::manifest_parser] Failed to load manifest directly. Will attempt auto-conversion from init-dir format.
Jun 28 11:38:33 qemux86-64 kanto-auto-deployer[500]: [2023-06-28T11:38:33Z INFO  kanto_auto_deployer] Already exists [seatservice-example]
Jun 28 11:38:33 qemux86-64 kanto-auto-deployer[500]: [2023-06-28T11:38:33Z WARN  kanto_auto_deployer::manifest_parser] Failed to load manifest directly. Will attempt auto-conversion from init-dir format.
Jun 28 11:38:33 qemux86-64 kanto-auto-deployer[500]: [2023-06-28T11:38:33Z INFO  kanto_auto_deployer] Already exists [sua]
Jun 28 11:38:33 qemux86-64 kanto-auto-deployer[500]: [2023-06-28T11:38:33Z WARN  kanto_auto_deployer::manifest_parser] Failed to load manifest directly. Will attempt auto-conversion from init-dir format.
Jun 28 11:38:33 qemux86-64 kanto-auto-deployer[500]: [2023-06-28T11:38:33Z INFO  kanto_auto_deployer] Already exists [vum]
Jun 28 11:38:33 qemux86-64 kanto-auto-deployer[500]: [2023-06-28T11:38:33Z INFO  kanto_auto_deployer] Running in daemon mode. Continuously monitoring "/data/var/containers/manifests"
Jun 28 11:38:33 qemux86-64 kanto-auto-deployer[500]: [2023-06-28T11:38:33Z INFO  kanto_auto_deployer::mqtt_listener] MQTT for daemon mode enabled. Will auto-disable whenever VUM takes over.
Jun 28 11:40:58 qemux86-64 kanto-auto-deployer[500]: [2023-06-28T11:40:58Z WARN  kanto_auto_deployer::fs_watcher] Getting terminated from MQTT!
Jun 28 11:40:58 qemux86-64 systemd[1]: kanto-auto-deployer.service: Deactivated successfully.
Jun 28 11:41:28 qemux86-64 systemd[1]: Kanto Auto Deployer was skipped because of a failed condition check (ConditionPathExists=/data/var/lib/kanto-auto-deployer/KAD.enabled).

Other