Overwrite (only) desired properties of module through layered deployment

[F-Joachim]

Question: Is it possible to only overwrite desired properties of a custom module without specifying the module (image name, createOptions, etc.) within the $edgeAgent block?

Info:

• iotedge version: 1.2.0

Our current deployment setup is that we have a basic deployment (priority 0) that contains basic settings for edgeAgent and edgeHub:

{
  "modulesContent": {
    "$edgeAgent": {
      "properties.desired": {
        "modules": {},
        "runtime": {
          "settings": {
            "minDockerVersion": "v1.25"
          },
          "type": "docker"
        },
        "schemaVersion": "1.0",
        "systemModules": {
          "edgeAgent": {
            "settings": {
              "image": "mcr.microsoft.com/azureiotedge-agent:1.2.0",
              "createOptions": "{\"ExposedPorts\":{\"9600/tcp\":{}},\"HostConfig\":{\"Binds\":[\"/srv/edgeAgent:/storage\"],\"PortBindings\":{\"9600/tcp\":[{\"HostPort\":\"8082\"}]}}}"
            },
            "type": "docker",
            "env": {
              "UpstreamProtocol": {
                "value": "AmqpWs"
              },
              "storageFolder": {
                "value": "/storage"
              },
              "SendRuntimeQualityTelemetry": {
                "value": "false"
              },
              "RuntimeLogLevel": {
                "value": "verbose"
              }
            }
          },
          "edgeHub": {
            "settings": {
              "image": "mcr.microsoft.com/azureiotedge-hub:1.2.0",
              "createOptions": "{\"ExposedPorts\":{\"9600/tcp\":{}},\"HostConfig\":{\"Binds\":[\"/srv/edgeHub:/storage\"],\"PortBindings\":{\"443/tcp\":[{\"HostPort\":\"443\"}],\"5671/tcp\":[{\"HostPort\":\"5671\"}],\"8883/tcp\":[{\"HostPort\":\"8883\"}],\"9600/tcp\":[{\"HostPort\":\"8081\"}]}}}"
            },
            "type": "docker",
            "env": {
              "UpstreamProtocol": {
                "value": "AmqpWs"
              },
              "storageFolder": {
                "value": "/storage"
              }
            },
            "status": "running",
            "restartPolicy": "always"
          }
        }
      }
    },
    "$edgeHub": {
      "properties.desired": {
        "routes": {
          "baseline-metrics": "FROM /messages/modules/baseline-metrics/* INTO $upstream"
        },
        "schemaVersion": "1.0",
        "storeAndForwardConfiguration": {
          "timeToLiveSecs": 7200
        }
      }
    }
  }
}

Additionally we have a use-case specific layered deployment (priority 10):

{
  "content": {
    "modulesContent": {
      "$edgeAgent": {
        "properties.desired.modules.module-A": {
          "settings": {
            "image": "<image name module-A>",
            "createOptions": ""
          },
          "type": "docker",
          "status": "running",
          "restartPolicy": "always",
          "version": "1.0"
        },
        "properties.desired.modules.module-B": {
          "settings": {
            "image": "<image name module-B>",
            "createOptions": ""
          },
          "type": "docker",
          "status": "running",
          "restartPolicy": "always",
          "version": "1.0"
        },
        "properties.desired.modules.module-C": {
          "settings": {
            "image": "<image name module-C>",
            "createOptions": ""
          },
          "type": "docker",
          "status": "running",
          "restartPolicy": "always",
          "version": "1.0"
        }
      },
      "module-A": {
        "properties.desired": {
          "my-property": "my-value"
        }
      }
    }
  }
}

For specific use cases we want to overwrite the desired property my-property of module-A in a separate layered deployment (priority 100) without specifying the module (image name etc.) within the $edgeAgent block, similar to:

{
  "content": {
    "modulesContent": {
      "$edgeAgent": {},
      "module-A": {
        "properties.desired": {
          "my-property": "use-another-value"
        }
      }
    }
  }
}

Can you please tell me if this is possible or how the deployment configuration has to be specified.

[chieftn]

Hi @F-Joachim, thank you for your inquiry. This strategy will function as you intend -- the contents of the deployment manifest stored in the $edgeAgent module twin will remain intact and custom module twin (e.g. module-A) will be updated to reflect the new value. This level of granular control is not available in the Azure portal can be achieved utilizing the command line interface to submit the value. The command parameters are available here: https://docs.microsoft.com/en-us/cli/azure/iot/edge/deployment?view=azure-cli-latest

[F-Joachim]

Thanks for your reply @chieftn. I have tried to create the three deployments as described here. But the deployments will not be applied, because of a validation error within the edgeAgent container:

<7> 2021-05-27 05:21:49.311 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Starting reconcile operation
<7> 2021-05-27 05:21:49.312 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient] - Making a Http call to unix:///var/run/iotedge/mgmt.sock to List modules
<7> 2021-05-27 05:21:49.312 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Connecting socket /var/run/iotedge/mgmt.sock
<7> 2021-05-27 05:21:49.313 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Connected socket /var/run/iotedge/mgmt.sock
<7> 2021-05-27 05:21:49.314 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Sending request http://mgmt.sock/modules?api-version=2020-07-07
<7> 2021-05-27 05:21:49.315 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Getting edge agent config...
<7> 2021-05-27 05:21:49.332 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Util.Uds.HttpUdsMessageHandler] - Response received OK
<7> 2021-05-27 05:21:49.333 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient] - Received a valid Http response from unix:///var/run/iotedge/mgmt.sock for List modules
<4> 2021-05-27 05:21:49.334 +00:00 [WRN] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Reconcile failed because of invalid configuration format
Microsoft.Azure.Devices.Edge.Agent.Core.ConfigSources.ConfigFormatException: Agent configuration format is invalid.
 ---> Newtonsoft.Json.JsonSerializationException: Could not find type in JObject.
   at Microsoft.Azure.Devices.Edge.Util.JsonEx.Get[T](JObject obj, String key) in /home/vsts/work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/JsonEx.cs:line 35
   at Microsoft.Azure.Devices.Edge.Agent.Core.Serde.TypeSpecificSerDe`1.TypeSpecificJsonConverter.ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/serde/TypeSpecificSerDe.cs:line 99
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.DeserializeConvertable(JsonConverter converter, JsonReader reader, Type objectType, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateDictionary(IDictionary dictionary, JsonReader reader, JsonDictionaryContract contract, JsonProperty containerProperty, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.ResolvePropertyAndCreatorValues(JsonObjectContract contract, JsonProperty containerProperty, JsonReader reader, Type objectType)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObjectUsingCreatorWithParameters(JsonReader reader, JsonObjectContract contract, JsonProperty containerProperty, ObjectConstructor`1 creator, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateNewObject(JsonReader reader, JsonObjectContract objectContract, JsonProperty containerMember, JsonProperty containerProperty, String id, Boolean& createdFromNonDefaultCreator)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
   at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonSerializer.Deserialize(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonConvert.DeserializeObject(String value, Type type, JsonSerializerSettings settings)
   at Newtonsoft.Json.JsonConvert.DeserializeObject[T](String value, JsonSerializerSettings settings)
   at Microsoft.Azure.Devices.Edge.Agent.Core.Serde.TypeSpecificSerDe`1.Deserialize[TU](String json) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/serde/TypeSpecificSerDe.cs:line 63
   at Microsoft.Azure.Devices.Edge.Agent.Core.Serde.TypeSpecificSerDe`1.Deserialize(String json) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/serde/TypeSpecificSerDe.cs:line 56
   at Microsoft.Azure.Devices.Edge.Agent.IoTHub.EdgeAgentConnection.UpdateDeploymentConfig(TwinCollection desiredProperties) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.IoTHub/EdgeAgentConnection.cs:line 359
   --- End of inner exception stack trace ---
   at Microsoft.Azure.Devices.Edge.Agent.IoTHub.EdgeAgentConnection.UpdateDeploymentConfig(TwinCollection desiredProperties) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.IoTHub/EdgeAgentConnection.cs:line 371
   at Microsoft.Azure.Devices.Edge.Agent.IoTHub.EdgeAgentConnection.<RefreshTwinAsync>b__32_0(Twin twin) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.IoTHub/EdgeAgentConnection.cs:line 267
   at Microsoft.Azure.Devices.Edge.Agent.Core.Agent.ReconcileAsync(CancellationToken token) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/Agent.cs:line 128
   at Microsoft.Azure.Devices.Edge.Agent.Core.Agent.ReconcileAsync(CancellationToken token) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/Agent.cs:line 128
   at Microsoft.Azure.Devices.Edge.Agent.Core.Agent.ReconcileAsync(CancellationToken token) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/Agent.cs:line 128
   at Microsoft.Azure.Devices.Edge.Agent.Core.Agent.ReconcileAsync(CancellationToken token) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/Agent.cs:line 128
   at Microsoft.Azure.Devices.Edge.Agent.Core.Agent.ReconcileAsync(CancellationToken token) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/Agent.cs:line 128
   at Microsoft.Azure.Devices.Edge.Agent.Core.Agent.ReconcileAsync(CancellationToken token) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/Agent.cs:line 128
   at Microsoft.Azure.Devices.Edge.Agent.Core.Agent.ReconcileAsync(CancellationToken token) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/Agent.cs:line 128
   at Microsoft.Azure.Devices.Edge.Agent.Core.Agent.ReconcileAsync(CancellationToken token) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/Agent.cs:line 128
   at Microsoft.Azure.Devices.Edge.Agent.Core.Agent.ReconcileAsync(CancellationToken token) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/Agent.cs:line 128
   at Microsoft.Azure.Devices.Edge.Agent.Core.Agent.ReconcileAsync(CancellationToken token) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/Agent.cs:line 128
<7> 2021-05-27 05:21:49.336 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.IoTHub.Reporters.IoTHubReporter] - Not updating reported properties as patch was found to be empty
<7> 2021-05-27 05:21:49.336 +00:00 [DBG] [Microsoft.Azure.Devices.Edge.Agent.Core.Agent] - Finished reconcile operation

The same error is also displayed in the Azure Portal:

The deployment configuration I used:

Deployment created with the following CLI command:

az iot edge deployment create \
                 --deployment-id "baseline" \
                 --content "<file-path-of-following-json>" \
                 --target-condition "deviceId!=''" \
                 --priority 0 \
                 --hub-name "my-fancy-iot-hub"

{
  "modulesContent": {
    "$edgeAgent": {
      "properties.desired": {
        "modules": {},
        "runtime": {
          "settings": {
            "minDockerVersion": "v1.25"
          },
          "type": "docker"
        },
        "schemaVersion": "1.0",
        "systemModules": {
          "edgeAgent": {
            "settings": {
              "image": "mcr.microsoft.com/azureiotedge-agent:1.2.0",
              "createOptions": "{\"ExposedPorts\":{\"9600/tcp\":{}},\"HostConfig\":{\"Binds\":[\"/srv/edgeAgent:/storage\"],\"PortBindings\":{\"9600/tcp\":[{\"HostPort\":\"8082\"}]}}}"
            },
            "type": "docker",
            "env": {
              "UpstreamProtocol": {
                "value": "AmqpWs"
              },
              "storageFolder": {
                "value": "/storage"
              },
              "SendRuntimeQualityTelemetry": {
                "value": "false"
              },
              "RuntimeLogLevel": {
                "value": "verbose"
              }
            }
          },
          "edgeHub": {
            "settings": {
              "image": "mcr.microsoft.com/azureiotedge-hub:1.2.0",
              "createOptions": "{\"ExposedPorts\":{\"9600/tcp\":{}},\"HostConfig\":{\"Binds\":[\"/srv/edgeHub:/storage\"],\"PortBindings\":{\"443/tcp\":[{\"HostPort\":\"443\"}],\"5671/tcp\":[{\"HostPort\":\"5671\"}],\"8883/tcp\":[{\"HostPort\":\"8883\"}],\"9600/tcp\":[{\"HostPort\":\"8081\"}]}}}"
            },
            "type": "docker",
            "env": {
              "UpstreamProtocol": {
                "value": "AmqpWs"
              },
              "storageFolder": {
                "value": "/storage"
              }
            },
            "status": "running",
            "restartPolicy": "always"
          }
        }
      }
    },
    "$edgeHub": {
      "properties.desired": {
        "routes": {
          "baseline-metrics": "FROM /messages/modules/baseline-metrics/* INTO $upstream"
        },
        "schemaVersion": "1.0",
        "storeAndForwardConfiguration": {
          "timeToLiveSecs": 7200
        }
      }
    }
  }
}

First layered deployment (contains default desired properties) created with the following CLI command:

az iot edge deployment create \
                 --deployment-id "default-use-case-deployment" \
                 --content "<file-path-of-following-json" \
                 --target-condition "tags.<use-case-name>" \
                 --priority 10 \
                 --layered true \
                 --hub-name "my-fancy-iot-hub"

{
  "modulesContent": {
    "$edgeAgent": {
      "properties.desired.modules.module-A": {
        "settings": {
          "image": "<image name module-A>",
          "createOptions": ""
        },
        "type": "docker",
        "status": "running",
        "restartPolicy": "always",
        "version": "1.0"
      }
    },
    "module-A": {
      "properties.desired": {
        "my-property": "my-value"
      }
    }
  }
}

Second layered deployment (contains device specific desired properties) created with the following CLI command:

az iot edge deployment create \
                 --deployment-id "specific-use-case-deployment" \
                 --content "<file-path-of-following-json" \
                 --target-condition "deviceId='device1'" \
                 --priority 100 \
                 --layered true \
                 --hub-name "my-fancy-iot-hub"

{
  "modulesContent": {
    "$edgeAgent": {
      "properties.desired.modules.module-A": {}
    },
    "module-A": {
      "properties.desired": {
        "my-property": "device-specific-value"
      }
    }
  }
}

Version information:

• iotedge: 1.2.0
• azureiotedge-agent: 1.2.0
• azureiotedge-hub: 1.2.0
• Azure CLI: 2.23.0

Please contact me, if you need more information. Thanks in advance for your reply.

[chieftn]

Thank you for the details -- I suspect the problem may reside with the second layered deployment -- specifically, under the $edgeAgent entry:

'properties.desired.modules.module-A': {} 

This line is basically telling the configuration/deployment engine to clear prior settings for the module -- which -- I suspect, is causing both the device issue and the portal issue. To address your initial question, I went through your scenario and was successful by leaving $edgeAgent empty (specifically: "$edgeAgent": {}). I, perhaps mistakenly, thought you had this field empty in your initial question.

I recommend setting the "module-A" entry to read (particularly in the second layered deployment):

"properties.desired.my-property": "device-specific-value"

This change will ensure that only my-property is changes. The entry you provided will set the module's entire properties.desired value.

[F-Joachim]

Thanks for your reply @chieftn. I have updated my configuration so that the $edgeAgent remains empty. This was actually my initial configuration, but after reading the Azure CLI examples I thought that I have to set an empty module block within the $edgeAgent block.

Now I get the callback from the module's SDK with the overwritten desired property. However, the Azure Portal says that the configuration is not applied.

[chieftn]

@F-Joachim, I noticed this too. I was planning to raise this as a separate issue as it seems to relate more to how the metric is calculated relative to the contents of the deployment.

[F-Joachim]

@chieftn, thanks for confirming the problem. Can you please let me know how I can track the issue?

[F-Joachim]

@chieftn, can you kindly give me an estimation of when the issue will be fixed. Thanks a lot.

[F-Joachim]

@chieftn Do you have any updates on this issue?

[veyalla]

While not directly answering your question, here is slightly different approach to achieve the desired outcome. It has the added benefit of not using up a deployment (which has a limit of 100 per IoT Hub) for device specific settings.

The key is to encode device unique information in a module’s desired properties section that is not targeted by any deployment.

Here are the steps:

1. Separate a module’s common and unique properties into two sections within the module’s twin. Example:

"module1": {
    "properties.desired": {
        "common": {
            "region": "us"
        },
        "unique": {
            "ip": "1.1.1.1"
        }
    }
}

2. After creating the device in IoT Hub, create a module identity and set the device specific information in the unique section via module twin update. This is how you make every device different without using an entire deployment.

az iot hub module-identity create -n hub1 -d d1 -m module1

az iot hub module-twin update -n hub1 -d d1 -m module1 --desired '{"unique":{"ip":"1.1.1.1"}}'

3. Now create an ADM deployment for a set of devices ensuring the unique section or its parent section is not targeted. Example:

{
    "id": "t",
    "priority": null,
    "targetCondition": "",
    "content": {
        "modulesContent": {
            "$edgeAgent": {
                "properties.desired": {
                    "modules": {
                        "module1": {
                            "settings": {
                                "image": "module1:latest",
                                "createOptions": ""
                            },
                            "type": "docker",
                            "status": "running",
                            "restartPolicy": "always",
                            "version": "1.0"
                        }
                    },
                    "runtime": {
                        "settings": {
                            "minDockerVersion": "v1.25"
                        },
                        "type": "docker"
                    },
                    "schemaVersion": "1.1",
                    "systemModules": {
                        "edgeAgent": {
                            "settings": {
                                "image": "mcr.microsoft.com/azureiotedge-agent:1.1",
                                "createOptions": ""
                            },
                            "type": "docker"
                        },
                        "edgeHub": {
                            "settings": {
                                "image": "mcr.microsoft.com/azureiotedge-hub:1.1",
                                "createOptions": "{\"HostConfig\":{\"PortBindings\":{\"443/tcp\":[{\"HostPort\":\"443\"}],\"5671/tcp\":[{\"HostPort\":\"5671\"}],\"8883/tcp\":[{\"HostPort\":\"8883\"}]}}}"
                            },
                            "type": "docker",
                            "status": "running",
                            "restartPolicy": "always"
                        }
                    }
                }
            },
            "$edgeHub": {
                "properties.desired": {
                    "routes": {},
                    "schemaVersion": "1.1",
                    "storeAndForwardConfiguration": {
                        "timeToLiveSecs": 7200
                    }
                }
            },
            "module1": {
+                "properties.desired.common": {
                    "region": "us"
                }
            }
        }
    },
    "labels": {},
    "metrics": {
        "queries": {},
        "results": {}
    },
    "etag": ""
}

With this approach the final desired properties that module1 will get when the device is provisioned will be what is shown in step 1.

You can now update the deployment for changing the common properties or update module twin directly to change the unique properties and things will behave as you'd expect.

[github-actions[bot]]

This issue is being marked as stale because it has been open for 30 days with no activity.

[scb01]

@F-Joachim @Paddy613

Apologies for the delay in addressing your question.

As documented here https://docs.microsoft.com/en-us/azure/iot-edge/module-edgeagent-edgehub?view=iotedge-2020-11 the edgeAgent section and a number of its child elements are required to specified as part of any deployment. So omitting those will not work.

For example, in the properties.desired section, you must specify schemaVersion, runtime.type etc as mentioned in the doc. Similarly, the properties.desired.systemModules section must contain the subsections for edgeAgent and edgeHub. The properties.desired.modules must contain all the modules you wish to have in your deployment and so on.

So for your scenario, I would recommend keeping the edgeAgent and edgeHub sections the same across the layered deployments. You can specify only the modules (i.e., module-A, module-B etc) and the desired properties that you want to set for those modules.

Let me know if you have further questions that I can help with.

[F-Joachim]

Thanks for your reply @scb01, but your answer does not address the problem that @chieftn and I identified. From a technical perspective, the deployment and overwriting of the desired properties works correctly. However, the problem is that the Azure Portal view is not correct. That was also confirmed by @chieftn.

Please let me know if you need any more information from me. Many thanks in advance.

[scb01]

@F-Joachim Yes, you are correct. My answer was addressing @Paddy613's question, albeit in a non-layered deployment scenario.

By the way, I was able to reproduce the issue that you and @chieftn identified. I will follow up with this internally and post back here when I have an update.

[F-Joachim]

Thanks for your reply @scb01. I look forward to hearing from you again.

[scb01]

Hello @F-Joachim

Thanks for your continued patience on this issue. I reproduced this in my environment by running multiple deployments. The interesting behavior I saw was that when I first created deployment test5, it showed 1 targeted and 1 applied as per my expectations. When I performed another deployment test6, it showed test6 as having 1 targeted and 1 applied, but changed test5 to 1 targeted and 0 applied which was unexpected. I followed up internally and here is what I found out from the team.

The metrics issue is by design. The portal collects the metrics by requesting a list of configurations that has information of which deployments are currently applied, not a history of deployment results, so any deployments that are overridden by another deployment would have applied equal to 0.

Please let me know if this explains what you are observing and if you have further questions.

[F-Joachim]

Hi @scb01

thanks a lot for your response. I am not sure we are talking about the same problem :-) What I have observed is, that the (layered) deployment that overrides a desired property of a lower prior deployment is marked targeted but not applied.

In your case it seems different: "[..] so any deployments that are overridden by another deployment would have applied equal to 0"

Let me try to explain the problem again with the help of the following image:

Baseline deployment What I want to achieve is that each edge device (Edge-1, Edge-2, ..., Edge-n) is equipped with a baseline deployment (blue), that contains the edgeAgent and edgeHub modules. Addressed is the deployment to each edge device by the generic target-condition deviceId!=''.

Use case layer Each use case layer (green and orange) contains a composition of modules describing an application for a certain use case. The same use case can be active on multiple edge devices. The deployment will be assigned by matching the custom tag on the edge device with the corresponding target-condition in the deployment.

Device layer In certain cases it is necessary that desired properties of the use case layer must be overridden (e.g. server name, connection properties etc.) . This will be done with a device specific layer (yellow and magenta), that targets the edge device by its deviceId.

Sample application

To demonstrate the concept described above, I've created a small demo application that you can easily deploy in your own development environment. I've created a Docker container that contains a tiny application that prints the value of the desired property demo. (For development purposes I use VirtualBox with Ubuntu 20.04 and IoT Edge version 1.2.3 as edge device.)

Deployment of the device layer

az iot edge deployment create \
    --deployment-id "device-layer" \
    --content ./layer-device.json \
    --target-condition "tags.deployment-demo=true" \
    --layered \
    --priority 100 \
    --hub-name <replace-with-name-of-your-iot-hub>

layer-device.json

{
  "modulesContent": {
    "$edgeAgent": {
    },
    "$edgeHub": {
    },
    "module-client": {
      "properties.desired": {
        "demo": {
          "my-useful-property": "overridden-value"
        }
      }
    }
  }
}

Deployment of the use case layer

az iot edge deployment create \
    --deployment-id "use-case-layer" \
    --content ./layer-use-case.json \
    --target-condition "tags.deployment-demo=true" \
    --layered \
    --priority 10 \
    --hub-name <replace-with-name-of-your-iot-hub>

layer-use-case.json

{
  "modulesContent": {
    "$edgeAgent": {
      "properties.desired.modules.module-client": {
        "settings": {
          "image": "fjoachim/simple-module-client:bd9280b"
        },
        "type": "docker",
        "status": "running",
        "restartPolicy": "always",
        "version": "1.0"
      }
    },
    "$edgeHub": {
    }
  },
  "module-client": {
    "properties.desired": {
      "demo": {
        "my-useful-property": "initial-value"
      }
    }
  }
}

Deployment of the baseline

az iot edge deployment create \
    --deployment-id "baseline" \
    --content ./baseline.json \
    --target-condition "deviceId!=''" \
    --priority 0 \
    --hub-name <replace-with-name-of-your-iot-hub>

baseline.json

{
  "modulesContent": {
    "$edgeAgent": {
      "properties.desired": {
        "modules": {
        },
        "runtime": {
          "settings": {
            "minDockerVersion": "v1.25"
          },
          "type": "docker"
        },
        "schemaVersion": "1.0",
        "systemModules": {
          "edgeAgent": {
            "settings": {
              "image": "mcr.microsoft.com/azureiotedge-agent:1.2.3",
              "createOptions": "{\"ExposedPorts\":{\"9600/tcp\":{}},\"HostConfig\":{\"Binds\":[\"/srv/edgeAgent:/storage\"],\"PortBindings\":{\"9600/tcp\":[{\"HostPort\":\"8082\"}]}}}"
            },
            "type": "docker",
            "env": {
              "UpstreamProtocol": {
                "value": "AmqpWs"
              },
              "storageFolder": {
                "value": "/storage"
              },
              "SendRuntimeQualityTelemetry": {
                "value": "false"
              }
            }
          },
          "edgeHub": {
            "settings": {
              "image": "mcr.microsoft.com/azureiotedge-hub:1.2.3",
              "createOptions": "{\"ExposedPorts\":{\"9600/tcp\":{}},\"HostConfig\":{\"Binds\":[\"/srv/edgeHub:/storage\"],\"PortBindings\":{\"443/tcp\":[{\"HostPort\":\"443\"}],\"5671/tcp\":[{\"HostPort\":\"5671\"}],\"8883/tcp\":[{\"HostPort\":\"8883\"}],\"9600/tcp\":[{\"HostPort\":\"8081\"}]}}}"
            },
            "type": "docker",
            "env": {
              "UpstreamProtocol": {
                "value": "AmqpWs"
              },
              "storageFolder": {
                "value": "/storage"
              }
            },
            "status": "running",
            "restartPolicy": "always"
          }
        }
      }
    },
    "$edgeHub": {
      "properties.desired": {
        "routes": {
        },
        "schemaVersion": "1.0",
        "storeAndForwardConfiguration": {
          "timeToLiveSecs": 7200
        }
      }
    }
  }
}

After the deployments have been successfully applied you can switch to the console of your edge device and run the following command to see the logs of the module module-client. As expected the container log shows the overridden value of the device layer, because of its higher priority.

$ docker logs module-client
IoT Hub responded to device twin operation with statusOK
Demo property changed to {"my-useful-property":"overridden-value"}

Back to the IoT Hub's deployment view in the Azure Portal we can notice that the layered deployment (that overrides a certain desired property) is marked as targeted but not applied:

I hope that I was able to illustrate the problem. Please let me know if you need further information.

[scb01]

Thanks @F-Joachim for the excellent level of detail in your response. I was able to exactly reproduce this issue on my side. In my environment, I have 3 devices (named mylnxbox, test2 and test3) and I have setup 2 devices (mylnxbox, test2) to have the deployment-demo tag. The baseline deployment targets all 3 devices, the use-case deployment targets only the 2 devices that have the deployment-demo tag and the device level deployment targets a specific device (targets mylnxbox). And as shown below, the device level deployment has 0 applied, even though the module logs show that the property has been overwritten.

Following up internally with the team that works on this API, we root caused the behavior to how the applied metric is calculated. The applied metric is calculated using this internal query

select deviceId from devices.modules where moduleId = '$edgeAgent' 
and configurations.[[device-layer]].status = 'Applied'

The device level deployment that you showed only changes the property on module-client and does not change anything on the $edgeAgent module. Since this query only looks at the data where the moduleId is set to $edgeAgent, it shows the applied count as 0.

Creating a custom metric with the following definition will show that the number of devices the deployment was applied on is 1.

select deviceId from devices.modules where moduleId = 'module-client' 
and configurations.[[device-layer]].status = 'Applied'

Here are some screenshots [Note: my custom module is named CSharpModule]

The custom metric definition

The targeted metric The applied metric and the custom metric

Unfortunately, this is the way that the metric on the portal is calculated and I'm not aware of any plans to change it. A way for you to get the visibility on # of devices applied is to create a custom metric and use that instead to track the deployment. I am adding @isyama, who works in the team that owns the layered deployment tool and metrics, so that he can chime in as well. Please do let me and @isyama know if there are further questions.

[F-Joachim]

Thanks @scb01 for the provided solution. I have implemented the device metric as you described and it worked as expected. I think I can use this approach as a temporary workaround but it would be fine if the system metric's query will be adjusted to cover the described case that only desired properties of the modules were overridden. In my opinion the system metric should not depend on which part of the module's configuration is overridden.

Consider the following examples:

Override environment variable of a module

Result: Deployment appears as applied in the Azure Portal view

{
  "modulesContent": {
    "$edgeAgent": {
      "properties.desired.modules.module-client.env.HOST": {
        "value": "db-server"
      }
    },
    "$edgeHub": {
    }
  }
}

Override desired property of a module

Result: Deployment appears as not applied in the Azure Portal view

{
  "modulesContent": {
    "$edgeAgent": {
    },
    "$edgeHub": {
    },
    "module-client": {
      "properties.desired": {
        "my-property": "default"
      }
    }
  }
}

[scb01]

@F-Joachim Good to hear that the custom metric approach is working for you.

I understand what you are saying about how the system metric should work. I will let @isyama work with his product team to look into your request and get back to you.

[isyama]

@F-Joachim Thank you for reporting the issue to us. I have created an internal tracking item for this request. We will review your request and come up with a holistic solution for you and other customers. Thanks again.

[F-Joachim]

@isyama Thanks for your reply. I look forward to hearing from you.

[isyama]

@F-Joachim After internal discussion with the feature team, unfortunately the current behavior is by design. We are also afraid of a potential risk of disrupting other customers’ businesses by changing the behavior. However, we have a long-term plan to improve the customer experience of the feature by addressing issues such as yours and also current design limitation. We cannot share much details about the plan right now, but we will be sure to let you know when it is ready for preview. Thank you again for contacting us.

[F-Joachim]

@isyama. Thanks for your reply.

That would be a very useful feature for us. But for the moment it's sufficient to have the custom device metric. I am looking forward to hearing from you. Please let me know if I can contribute to this issue in any way. Thanks in advance.

[isyama]

@F-Joachim I unassigned the issue from myself by mistake. I will assign the issue to Sudeep (@sudeepster) who has more context on the project. Thanks.