New tedge twin (inventory) MQTT topic

[reubenmiller]

Is your feature request related to a problem? Please describe.

Support new tedge telemetry type called twin (in Cumulocity IoT this is called inventory managed object updates) to add json data to the device, child device or service's digital twin in the Cloud.

Describe the solution you'd like

New topic following the tedge MQTT V1 interface to allow users to publish additional information to the entity (e.g. device, child device or service).

Topic name

te/{identity}/twin/<type>

Payload (json, retain=true)

{
    "family": "Debian",
    "version": 11
}

MQTT messages published to te/{identity}/twin/<type> should be published with the retained flag to allow other components to more easily observe the information.

Cloud mapper initialization (to be validated)

When the tedge-mapper starts up for a given cloud, it should run the following routine to process the initial values after any registration process has finished:

1. Read the inventory.json file, load the values into an in-memory model
2. Subscribe to the data topic using the MQTT topic pattern {identifier}/twin/+ and merge the results. The values received via MQTT take precedence over any duplicated values from the inventory.json file
3. Publish the inventory changes to the relevant cloud in one message (if possible)
4. Process all subsequent messages received on the {identifier}/twin/+ MQTT topic as single message to the relevant cloud

Open questions

• How to handle inventory.json? Previously the inventory.json file was used to publish static information to the cloud, however it was not that useful as it would only be read on startup and it also required users to have file access to the file (amongst other limitations).
  • Option 1: Deprecate the inventory.json file. Use a feature flag to deactivate/ignore the file.
  • Option 2: Use the inventory.json value as the persistence layer. Any messages published to the /data/ topics will be merged into the inventory.json before publishing

Describe alternatives you've considered

Additional context

A draft PR has already been created to include the following items to assist with the development:

• Documentation about tedge topic to c8y mapping for the new /twin/ topic
• System tests for main and child devices

Related tickets

• https://github.com/thin-edge/thin-edge.io/issues/1935
• https://github.com/thin-edge/thin-edge.io/issues/1296

[didier-wenzek]

@gligorisaev See this comment for the testing plan: https://github.com/thin-edge/thin-edge.io/pull/2280#issuecomment-1772255554

[gligorisaev]

QA has thoroughly checked the feature and here are the results:

• [x] Test for ticket exists in the test suite.
• tests/RobotFramework/tests/cumulocity/telemetry/child_device_telemetry.robot
• tests/RobotFramework/tests/cumulocity/telemetry/thin-edge_device_telemetry.robot
• tests/RobotFramework/tests/tedge/call_tedge_config_list.robot
• tests/RobotFramework/tests/MQTT_health_check/MQTT_health_endpoints.robot
• tests/RobotFramework/tests/MQTT_health_check/health_tedge-mapper-collectd.robot
• tests/RobotFramework/tests/cumulocity/service_monitoring/service_monitoring.robot
• tests/RobotFramework/tests/cumulocity/software_management/software.robot
• tests/RobotFramework/tests/cumulocity/supported_operations/mapper-publishing-agent-supported-ops.robot
• tests/RobotFramework/tests/tedge/call_tedge_config_list.robot
• tests/RobotFramework/tests/tedge_to_te_converter/convert_tedge_topics_to_te_topics.robot
• [x] QA has tested the bug and could not reproduce it anymore.