Nested IoT Edge fails to start on child device using IP addresses as hostnames

patrikSohlman commented 3 years ago

Following this guide, I am trying to create a deployment of nested IoT Edge devices following this guide. I have configured the parent-child relationship in the Azure portal and created certificates based on Create test certificates.

The parent device is working but the child device fails in starting the deamon as it can not fetch device information from the parent.

I am using IP addresses as hostnames as that is explicitly stated in the guide as a viable option, as long as you are consistent thorugh your nested structure.

Expected Behavior

Child device should be able to pull modules from $upstream and communicate to parent

Current Behavior

The child device fails with the error The daemon could not start up successfully: Could not retrieve device information

Click here

``` Jul 08 13:55:25 iot-gate-imx8 systemd[1]: Started Azure IoT Identity Service. Jul 08 13:55:25 iot-gate-imx8 aziot-identityd[11471]: 2021-07-08T13:55:25Z [INFO] - Starting service... Jul 08 13:55:25 iot-gate-imx8 aziot-identityd[11471]: 2021-07-08T13:55:25Z [INFO] - Version - 1.2.2 Jul 08 13:55:25 iot-gate-imx8 aziot-identityd[11471]: 2021-07-08T13:55:25Z [INFO] - Provisioning starting. Reason: Startup Jul 08 13:55:25 iot-gate-imx8 aziot-identityd[11471]: 2021-07-08T13:55:25Z [INFO] - Updated device info for red-iot-layer2-edge. Jul 08 13:55:25 iot-gate-imx8 aziot-identityd[11471]: 2021-07-08T13:55:25Z [INFO] - Provisioning complete. Jul 08 13:55:25 iot-gate-imx8 aziot-identityd[11471]: 2021-07-08T13:55:25Z [INFO] - Identity reconciliation started. Reason: Startup Jul 08 13:55:25 iot-gate-imx8 aziot-keyd[11163]: 2021-07-08T13:55:25Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"} Jul 08 13:55:25 iot-gate-imx8 aziot-keyd[11163]: 2021-07-08T13:55:25Z [INFO] - --> 200 {"content-type": "application/json"} Jul 08 13:55:25 iot-gate-imx8 aziot-keyd[11163]: 2021-07-08T13:55:25Z [INFO] - <-- POST /sign?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "398"} Jul 08 13:55:25 iot-gate-imx8 aziot-keyd[11163]: 2021-07-08T13:55:25Z [INFO] - --> 200 {"content-type": "application/json"} Jul 08 13:55:25 iot-gate-imx8 aziot-identityd[11471]: 2021-07-08T13:55:25Z [INFO] - Could not reconcile Identities with current device data. Reprovisioning. Jul 08 13:55:25 iot-gate-imx8 aziot-identityd[11471]: 2021-07-08T13:55:25Z [INFO] - Updated device info for red-iot-layer2-edge. Jul 08 13:55:25 iot-gate-imx8 aziot-keyd[11163]: 2021-07-08T13:55:25Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"} Jul 08 13:55:25 iot-gate-imx8 aziot-keyd[11163]: 2021-07-08T13:55:25Z [INFO] - --> 200 {"content-type": "application/json"} Jul 08 13:55:25 iot-gate-imx8 aziot-keyd[11163]: 2021-07-08T13:55:25Z [INFO] - <-- POST /sign?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "398"} Jul 08 13:55:25 iot-gate-imx8 aziot-keyd[11163]: 2021-07-08T13:55:25Z [INFO] - --> 200 {"content-type": "application/json"} Jul 08 13:55:25 iot-gate-imx8 aziot-identityd[11471]: 2021-07-08T13:55:25Z [ERR!] - Failed to provision with IoT Hub, and no valid device backup was found: Hub client error Jul 08 13:55:25 iot-gate-imx8 aziot-identityd[11471]: 2021-07-08T13:55:25Z [ERR!] - service encountered an error Jul 08 13:55:25 iot-gate-imx8 aziot-identityd[11471]: 2021-07-08T13:55:25Z [ERR!] - caused by: Hub client error Jul 08 13:55:25 iot-gate-imx8 aziot-identityd[11471]: 2021-07-08T13:55:25Z [ERR!] - caused by: expected value at line 1 column 1 Jul 08 13:55:25 iot-gate-imx8 aziot-identityd[11471]: 2021-07-08T13:55:25Z [ERR!] - 0: Jul 08 13:55:25 iot-gate-imx8 aziot-identityd[11471]: 1: Jul 08 13:55:25 iot-gate-imx8 systemd[1]: aziot-identityd.service: Main process exited, code=exited, status=1/FAILURE Jul 08 13:55:25 iot-gate-imx8 systemd[1]: aziot-identityd.service: Failed with result 'exit-code'. Jul 08 13:55:30 iot-gate-imx8 aziot-edged[11149]: 2021-07-08T13:55:30Z [INFO] - Obtaining edge device provisioning data... Jul 08 13:55:30 iot-gate-imx8 systemd[1]: aziot-identityd.service: Service RestartSec=5s expired, scheduling restart. Jul 08 13:55:30 iot-gate-imx8 systemd[1]: aziot-identityd.service: Scheduled restart job, restart counter is at 51. Jul 08 13:55:30 iot-gate-imx8 systemd[1]: Stopped Azure IoT Identity Service. Jul 08 13:55:30 iot-gate-imx8 aziot-edged[11149]: 2021-07-08T13:55:30Z [WARN] - The daemon could not start up successfully: Could not retrieve device information Jul 08 13:55:30 iot-gate-imx8 aziot-edged[11149]: 2021-07-08T13:55:30Z [WARN] - caused by: HTTP request error Jul 08 13:55:30 iot-gate-imx8 aziot-edged[11149]: 2021-07-08T13:55:30Z [WARN] - caused by: connection error: Connection reset by peer (os error 104) Jul 08 13:55:30 iot-gate-imx8 aziot-edged[11149]: 2021-07-08T13:55:30Z [WARN] - Requesting device reprovision. Jul 08 13:55:30 iot-gate-imx8 aziot-edged[11149]: 2021-07-08T13:55:30Z [WARN] - The reprovisioning operation failed ```

Steps to Reproduce

Provide a detailed set of steps to reproduce the bug.

Generate root cert following the guide stated above
Create IoT Edge device CA certificates for both parent and child.
Set up the relationship in the Azure portal
Install certs and configure /etc/aziot/config.toml
set hostname in parent config.tomlto "local-ip-address-of-parent"
Follow guide and set up registry and APIProxy modules on parent
set parent_hostname and hostname in child to "local-ip-address-of-parent" and "local-ip-address-of-child" respectively.
Apply configs

config.toml on child

Click here

```bash # ============================================================================== # Hostname # ============================================================================== # # Uncomment the next line to override the default hostname of this device. # hostname = "192.168.1.101" # ============================================================================== # Parent hostname # ============================================================================== # # If this is a Nested Edge device, uncomment the next line to set the # parent hostname of this device. # parent_hostname = "192.168.1.90" # ============================================================================== # Trust bundle cert # ============================================================================== # # If you have any trusted CA certificates required for Edge module communication, # uncomment the next line and set the value to a file URI for # the path of the file. # trust_bundle_cert = "file:///var/secrets/aziot/certs/red-iot-cert-test.root.ca.cert.pem" # ============================================================================== # Provisioning # ============================================================================== # Optional auto reprovisioning mode # ------------------------------------ # This property specifies the conditions under which the device attempts to # automatically reprovision with the cloud. It is ignored if the device has # been provisioned manually. One of the following values can be set: # Dynamic: Reprovision when the device detects that it may have # been moved from one IoT Hub to another. This is the default. # AlwaysOnStartup: Reprovision when the device is rebooted or a crash causes # the daemon(s) to restart. # OnErrorOnly: Never trigger device reprovisioning automatically. # Device reprovisioning only occurs as fallback, if the device # is unable to connect to IoT Hub during identity provisioning # due to connectivity errors. This fallback behavior is implicit # in Dynamic and AlwaysOnStartup modes as well. # # auto_reprovisioning_mode = Dynamic # # Provisioning configuration # -------------------------- # Uncomment one block and replace the stub values with yours. ## Manual provisioning with connection string # [provisioning] source = "manual" connection_string = "" ## Manual provisioning with symmetric key # [provisioning] # source = "manual" # iothub_hostname = "example.azure-devices.net" # device_id = "my-device" # # [provisioning.authentication] # method = "sas" # # device_id_pk = { value = "YXppb3QtaWRlbnRpdHktc2VydmljZXxhemlvdC1pZGU=" } # inline key (base64), or... # device_id_pk = { uri = "file:///var/secrets/device-id.key" } # file URI, or... # device_id_pk = { uri = "pkcs11:slot-id=0;object=device%20id?pin-value=1234" } # PKCS#11 URI ## Manual provisioning with X.509 certificate # [provisioning] # source = "manual" # iothub_hostname = "example.azure-devices.net" # device_id = "my-device" # # [provisioning.authentication] # method = "x509" # # identity_cert = "file:///var/secrets/device-id.pem" # file URI, or... # identity_cert = { method = "est", common_name = "my-device" } # dynamically issued via EST, or... # identity_cert = { method = "local_ca", common_name = "my-device" } # dynamically issued by a local CA # # identity_pk = "file:///var/secrets/device-id.key.pem" # file URI, or... # identity_pk = "pkcs11:slot-id=0;object=device%20id?pin-value=1234" # PKCS#11 URI ## DPS provisioning with symmetric key # [provisioning] # source = "dps" # global_endpoint = "https://global.azure-devices-provisioning.net" # id_scope = "0ab1234C5D6" # # [provisioning.attestation] # method = "symmetric_key" # registration_id = "my-device" # # symmetric_key = { value = "YXppb3QtaWRlbnRpdHktc2VydmljZXxhemlvdC1pZGVudGl0eS1zZXJ2aWNlfGF6aW90LWlkZW50aXR5LXNlcg==" } # inline key (base64), or... # symmetric_key = { uri = "file:///var/secrets/device-id.key" } # file URI, or... # symmetric_key = { uri = "pkcs11:slot-id=0;object=device%20id?pin-value=1234" } # PKCS#11 URI ## DPS provisioning with X.509 certificate # [provisioning] # source = "dps" # global_endpoint = "https://global.azure-devices-provisioning.net" # id_scope = "0ab1234C5D6" # # [provisioning.attestation] # method = "x509" # registration_id = "my-device" # # identity_cert = "file:///var/secrets/device-id.pem" # file URI, or... # identity_cert = { method = "est", common_name = "my-device" } # dynamically issued via EST, or... # identity_cert = { method = "local_ca", common_name = "my-device" } # dynamically issued by a local CA # # identity_pk = "file:///var/secrets/device-id.key.pem" # file URI, or... # identity_pk = "pkcs11:slot-id=0;object=device%20id?pin-value=1234" # PKCS#11 URI ## DPS provisioning with TPM # [provisioning] # source = "dps" # global_endpoint = "https://global.azure-devices-provisioning.net" # id_scope = "0ab1234C5D6" # # [provisioning.attestation] # method = "tpm" # registration_id = "my-device" # ============================================================================== # Cert issuance # ============================================================================== # # If you configured any dynamically issued certs above, uncomment the block # corresponding to the issuance method and replace the stub values with yours. # Cert issuance via EST # --------------------- # [cert_issuance.est] # trusted_certs = [ # "file:///var/secrets/est-id-ca.pem", # ] # # [cert_issuance.est.auth] # username = "estuser" # password = "estpwd" # # # EST ID cert already on device, or... # identity_cert = "file:///var/secrets/est-id.pem" # # identity_pk = "file:///var/secrets/est-id.key.pem" # file URI, or... # identity_pk = "pkcs11:slot-id=0;object=est-id?pin-value=1234" # PKCS#11 URI # # # EST ID cert requested via EST bootstrap ID cert # bootstrap_identity_cert = "file:///var/secrets/est-bootstrap-id.pem" # # bootstrap_identity_pk = "file:///var/secrets/est-bootstrap-id.key.pem" # file URI, or... # bootstrap_identity_pk = "pkcs11:slot-id=0;object=est-bootstrap-id?pin-value=1234" # PKCS#11 URI # # [cert_issuance.est.urls] # default = "https://example.org/.well-known/est" # Cert issuance via local CA # -------------------------- # [cert_issuance.local_ca] # cert = "file:///var/secrets/local-ca.pem" # # pk = "file:///var/secrets/local-ca.key.pem" # file URI, or... # pk = "pkcs11:slot-id=0;object=local-ca?pin-value=1234" # PKCS#11 URI # ============================================================================== # PKCS#11 # ============================================================================== # # If you used any PKCS#11 URIs above, uncomment the next block and # replace the stub values with your PKCS#11 configuration. # [aziot_keys] # pkcs11_lib_path = "/usr/lib/libmypkcs11.so" # pkcs11_base_slot = "pkcs11:slot-id=0?pin-value=1234" # ============================================================================== # Default Edge Agent # ============================================================================== # # If you need to override the parameters of the default Edge Agent module, # uncomment this section and replace the values in this section with your own. # # Note that the agent.config.createOptions field is specified as # a TOML inline table. This format looks similar to JSON but it is not JSON. # See https://toml.io/en/v1.0.0#inline-table for documentation. [agent] name = "edgeAgent" type = "docker" # imagePullPolicy = "..." # "on-create" or "never". Defaults to "on-create" [agent.config] image = "192.168.1.90:443/azureiotedge-agent:1.2" # createOptions = { HostConfig = { Binds = ["/iotedge/storage:/iotedge/storage"] } } # [agent.config.auth] # serveraddress = "example.azurecr.io" # username = "username" # password = "password" # [agent.env] # "RuntimeLogLevel" = "debug" # "UpstreamProtocol" = "AmqpWs" # "storageFolder" = "/iotedge/storage" # ============================================================================== # Daemon management and workload API endpoints # ============================================================================== # # If you need to override the management and workload API endpoints, # uncomment this section and replace the values in this section with your own. # [connect] # workload_uri = "unix:///var/run/iotedge/workload.sock" # management_uri = "unix:///var/run/iotedge/mgmt.sock" # # [listen] # workload_uri = "fd://aziot-edged.workload.socket" # management_uri = "fd://aziot-edged.mgmt.socket" # ============================================================================== # Edge Agent watchdog # ============================================================================== # # If you need to override the default Edge Agent watchdog settings, # uncomment this section and replace the values with your own. # # [watchdog] # max_retries = "infinite" # the string "infinite" or a positive integer. Defaults to "infinite" # ============================================================================== # Edge CA certificate # ============================================================================== # # If you have your own Edge CA certificate that you want all module certificates # to be issued by, uncomment this section and replace the values with your own. # [edge_ca] cert = "file:///var/secrets/aziot/certs/iot-edge-device-ca-layer2-full-chain.cert.pem" pk = "file:///var/secrets/aziot/certs/iot-edge-device-ca-layer2.key.pem" # ============================================================================== # Edge CA certificate (Quickstart) # ============================================================================== # # If you do not have your own Edge CA certificate that you want all # module certificates to be issued by, you can uncomment this section and # set the number of days for the lifetime of the auto-generated # self-signed Edge CA certificate. Defaults to 90 days. # # SECURITY NOTE: this setting is NOT recommended for production usage. Please # configure your own Edge CA certificate in the Edge CA certificate section # above instead. # # [edge_ca] # auto_generated_edge_ca_expiry_days = 90 # ============================================================================== # Moby runtime # ============================================================================== # # If you need to override the default Moby runtime configuration, # uncomment this section and replace the values in this section with your own. # [moby_runtime] # uri = "unix:///var/run/docker.sock" # network = "azure-iot-edge" ```

config.toml on parent

Click here

```bash # ============================================================================== # Hostname # ============================================================================== # # Uncomment the next line to override the default hostname of this device. # hostname = "192.168.1.90" # ============================================================================== # Parent hostname # ============================================================================== # # If this is a Nested Edge device, uncomment the next line to set the # parent hostname of this device. # # parent_hostname = "my-parent-device" # ============================================================================== # Trust bundle cert # ============================================================================== # # If you have any trusted CA certificates required for Edge module communication, # uncomment the next line and set the value to a file URI for # the path of the file. # trust_bundle_cert = "file:///var/secrets/aziot/certs/red-iot-cert-test.root.ca.cert.pem" # ============================================================================== # Provisioning # ============================================================================== # Optional auto reprovisioning mode # ------------------------------------ # This property specifies the conditions under which the device attempts to # automatically reprovision with the cloud. It is ignored if the device has # been provisioned manually. One of the following values can be set: # Dynamic: Reprovision when the device detects that it may have # been moved from one IoT Hub to another. This is the default. # AlwaysOnStartup: Reprovision when the device is rebooted or a crash causes # the daemon(s) to restart. # OnErrorOnly: Never trigger device reprovisioning automatically. # Device reprovisioning only occurs as fallback, if the device # is unable to connect to IoT Hub during identity provisioning # due to connectivity errors. This fallback behavior is implicit # in Dynamic and AlwaysOnStartup modes as well. # # auto_reprovisioning_mode = Dynamic # # Provisioning configuration # -------------------------- # Uncomment one block and replace the stub values with yours. # Manual provisioning with connection string [provisioning] source = "manual" connection_string = "" ## Manual provisioning with symmetric key # [provisioning] # source = "manual" # iothub_hostname = "example.azure-devices.net" # device_id = "my-device" # # [provisioning.authentication] # method = "sas" # # device_id_pk = { value = "YXppb3QtaWRlbnRpdHktc2VydmljZXxhemlvdC1pZGU=" } # inline key (base64), or... # device_id_pk = { uri = "file:///var/secrets/device-id.key" } # file URI, or... # device_id_pk = { uri = "pkcs11:slot-id=0;object=device%20id?pin-value=1234" } # PKCS#11 URI ## Manual provisioning with X.509 certificate # [provisioning] # source = "manual" # iothub_hostname = "example.azure-devices.net" # device_id = "my-device" # # [provisioning.authentication] # method = "x509" # # identity_cert = "file:///var/secrets/device-id.pem" # file URI, or... # identity_cert = { method = "est", common_name = "my-device" } # dynamically issued via EST, or... # identity_cert = { method = "local_ca", common_name = "my-device" } # dynamically issued by a local CA # # identity_pk = "file:///var/secrets/device-id.key.pem" # file URI, or... # identity_pk = "pkcs11:slot-id=0;object=device%20id?pin-value=1234" # PKCS#11 URI ## DPS provisioning with symmetric key # [provisioning] # source = "dps" # global_endpoint = "https://global.azure-devices-provisioning.net" # id_scope = "0ab1234C5D6" # # [provisioning.attestation] # method = "symmetric_key" # registration_id = "my-device" # # symmetric_key = { value = "YXppb3QtaWRlbnRpdHktc2VydmljZXxhemlvdC1pZGVudGl0eS1zZXJ2aWNlfGF6aW90LWlkZW50aXR5LXNlcg==" } # inline key (base64), or... # symmetric_key = { uri = "file:///var/secrets/device-id.key" } # file URI, or... # symmetric_key = { uri = "pkcs11:slot-id=0;object=device%20id?pin-value=1234" } # PKCS#11 URI ## DPS provisioning with X.509 certificate # [provisioning] # source = "dps" # global_endpoint = "https://global.azure-devices-provisioning.net" # id_scope = "0ab1234C5D6" # # [provisioning.attestation] # method = "x509" # registration_id = "my-device" # # identity_cert = "file:///var/secrets/device-id.pem" # file URI, or... # identity_cert = { method = "est", common_name = "my-device" } # dynamically issued via EST, or... # identity_cert = { method = "local_ca", common_name = "my-device" } # dynamically issued by a local CA # # identity_pk = "file:///var/secrets/device-id.key.pem" # file URI, or... # identity_pk = "pkcs11:slot-id=0;object=device%20id?pin-value=1234" # PKCS#11 URI ## DPS provisioning with TPM # [provisioning] # source = "dps" # global_endpoint = "https://global.azure-devices-provisioning.net" # id_scope = "0ab1234C5D6" # # [provisioning.attestation] # method = "tpm" # registration_id = "my-device" # ============================================================================== # Cert issuance # ============================================================================== # # If you configured any dynamically issued certs above, uncomment the block # corresponding to the issuance method and replace the stub values with yours. # Cert issuance via EST # --------------------- # [cert_issuance.est] # trusted_certs = [ # "file:///var/secrets/est-id-ca.pem", # ] # # [cert_issuance.est.auth] # username = "estuser" # password = "estpwd" # # # EST ID cert already on device, or... # identity_cert = "file:///var/secrets/est-id.pem" # # identity_pk = "file:///var/secrets/est-id.key.pem" # file URI, or... # identity_pk = "pkcs11:slot-id=0;object=est-id?pin-value=1234" # PKCS#11 URI # # # EST ID cert requested via EST bootstrap ID cert # bootstrap_identity_cert = "file:///var/secrets/est-bootstrap-id.pem" # # bootstrap_identity_pk = "file:///var/secrets/est-bootstrap-id.key.pem" # file URI, or... # bootstrap_identity_pk = "pkcs11:slot-id=0;object=est-bootstrap-id?pin-value=1234" # PKCS#11 URI # # [cert_issuance.est.urls] # default = "https://example.org/.well-known/est" # Cert issuance via local CA # -------------------------- # [cert_issuance.local_ca] # cert = "file:///var/secrets/local-ca.pem" # # pk = "file:///var/secrets/local-ca.key.pem" # file URI, or... # pk = "pkcs11:slot-id=0;object=local-ca?pin-value=1234" # PKCS#11 URI # ============================================================================== # PKCS#11 # ============================================================================== # # If you used any PKCS#11 URIs above, uncomment the next block and # replace the stub values with your PKCS#11 configuration. # [aziot_keys] # pkcs11_lib_path = "/usr/lib/libmypkcs11.so" # pkcs11_base_slot = "pkcs11:slot-id=0?pin-value=1234" # ============================================================================== # Default Edge Agent # ============================================================================== # # If you need to override the parameters of the default Edge Agent module, # uncomment this section and replace the values in this section with your own. # # Note that the agent.config.createOptions field is specified as # a TOML inline table. This format looks similar to JSON but it is not JSON. # See https://toml.io/en/v1.0.0#inline-table for documentation. # [agent] # name = "edgeAgent" # type = "docker" # imagePullPolicy = "..." # "on-create" or "never". Defaults to "on-create" # [agent.config] # image = "mcr.microsoft.com/azureiotedge-agent:1.2" # createOptions = { HostConfig = { Binds = ["/iotedge/storage:/iotedge/storage"] } } # [agent.config.auth] # serveraddress = "example.azurecr.io" # username = "username" # password = "password" # [agent.env] # "RuntimeLogLevel" = "debug" # "UpstreamProtocol" = "AmqpWs" # "storageFolder" = "/iotedge/storage" # ============================================================================== # Daemon management and workload API endpoints # ============================================================================== # # If you need to override the management and workload API endpoints, # uncomment this section and replace the values in this section with your own. # [connect] # workload_uri = "unix:///var/run/iotedge/workload.sock" # management_uri = "unix:///var/run/iotedge/mgmt.sock" # # [listen] # workload_uri = "fd://aziot-edged.workload.socket" # management_uri = "fd://aziot-edged.mgmt.socket" # ============================================================================== # Edge Agent watchdog # ============================================================================== # # If you need to override the default Edge Agent watchdog settings, # uncomment this section and replace the values with your own. # # [watchdog] # max_retries = "infinite" # the string "infinite" or a positive integer. Defaults to "infinite" # ============================================================================== # Edge CA certificate # ============================================================================== # # If you have your own Edge CA certificate that you want all module certificates # to be issued by, uncomment this section and replace the values with your own. # [edge_ca] cert = "file:///var/secrets/aziot/certs/iot-edge-device-ca-layer3-full-chain.cert.pem" pk = "file:///var/secrets/aziot/certs/iot-edge-device-ca-layer3.key.pem" # ============================================================================== # Edge CA certificate (Quickstart) # ============================================================================== # # If you do not have your own Edge CA certificate that you want all # module certificates to be issued by, you can uncomment this section and # set the number of days for the lifetime of the auto-generated # self-signed Edge CA certificate. Defaults to 90 days. # # SECURITY NOTE: this setting is NOT recommended for production usage. Please # configure your own Edge CA certificate in the Edge CA certificate section # above instead. # # [edge_ca] # auto_generated_edge_ca_expiry_days = 90 # ============================================================================== # Moby runtime # ============================================================================== # # If you need to override the default Moby runtime configuration, # uncomment this section and replace the values in this section with your own. # [moby_runtime] # uri = "unix:///var/run/docker.sock" # network = "azure-iot-edge" ```

Context (Environment)

Output of `iotedge check`

Click here

``` Configuration checks (aziot-identity-service) --------------------------------------------- √ keyd configuration is well-formed - OK √ certd configuration is well-formed - OK √ tpmd configuration is well-formed - OK √ identityd configuration is well-formed - OK √ daemon configurations up-to-date with config.toml - OK √ identityd config toml file specifies a valid hostname - OK √ host time is close to reference time - OK √ preloaded certificates are valid - OK √ keyd is running - OK √ certd is running - OK √ identityd is running - OK √ read all preloaded certificates from the Certificates Service - OK √ read all preloaded key pairs from the Keys Service - OK √ ensure all preloaded certificates match preloaded private keys with the same ID - OK Connectivity checks (aziot-identity-service) -------------------------------------------- √ host can connect to and perform TLS handshake with iothub AMQP port - OK √ host can connect to and perform TLS handshake with iothub HTTPS / WebSockets port - OK √ host can connect to and perform TLS handshake with iothub MQTT port - OK Configuration checks -------------------- √ aziot-edged configuration is well-formed - OK √ configuration up-to-date with config.toml - OK √ container engine is installed and functional - OK √ configuration has correct parent_hostname - OK × configuration has correct URIs for daemon mgmt endpoint - Error Unable to find image '192.168.1.90/azureiotedge-diagnostics:1.2.3' locally 1.2.3: Pulling from azureiotedge-diagnostics ed6dc9c66f7c: Pulling fs layer 15c11899c85b: Pulling fs layer 86ebe93eb4a1: Pulling fs layer 3c82b9816bd4: Pulling fs layer fefaff19cc6b: Pulling fs layer 8583e3308d73: Pulling fs layer 2bdf6b22f628: Pulling fs layer 4e3c467c1b79: Pulling fs layer 5be81669f54b: Pulling fs layer 3c82b9816bd4: Waiting fefaff19cc6b: Waiting 8583e3308d73: Waiting 2bdf6b22f628: Waiting 4e3c467c1b79: Waiting 5be81669f54b: Waiting 86ebe93eb4a1: Verifying Checksum 86ebe93eb4a1: Download complete 15c11899c85b: Verifying Checksum 15c11899c85b: Download complete fefaff19cc6b: Verifying Checksum fefaff19cc6b: Download complete ed6dc9c66f7c: Verifying Checksum ed6dc9c66f7c: Download complete 2bdf6b22f628: Verifying Checksum 2bdf6b22f628: Download complete 3c82b9816bd4: Verifying Checksum 3c82b9816bd4: Download complete 4e3c467c1b79: Verifying Checksum 4e3c467c1b79: Download complete 5be81669f54b: Verifying Checksum 5be81669f54b: Download complete 8583e3308d73: Verifying Checksum 8583e3308d73: Download complete ed6dc9c66f7c: Pull complete 15c11899c85b: Pull complete 86ebe93eb4a1: Pull complete 3c82b9816bd4: Pull complete fefaff19cc6b: Pull complete 8583e3308d73: Pull complete 2bdf6b22f628: Pull complete 4e3c467c1b79: Pull complete 5be81669f54b: Pull complete Digest: sha256:7a03ca253dc3cef0767b0c13703e7c9b2b4c662c93133e5be1b22c4e15ecadf8 Status: Downloaded newer image for 192.168.1.90/azureiotedge-diagnostics:1.2.3 SocketError - SocketErrorCode (AccessDenied) : Permission denied /var/run/iotedge/mgmt.sock One or more errors occurred. (Got bad response: ) √ container time is close to host time - OK ‼ DNS server - Warning Container engine is not configured with DNS server setting, which may impact connectivity to IoT Hub. Please see https://aka.ms/iotedge-prod-checklist-dns for best practices. You can ignore this warning if you are setting DNS server per module in the Edge deployment. ‼ production readiness: container engine - Warning Device is not using a production-supported container engine (moby-engine). Please see https://aka.ms/iotedge-prod-checklist-moby for details. ‼ production readiness: logs policy - Warning Container engine is not configured to rotate module logs which may cause it run out of disk space. Please see https://aka.ms/iotedge-prod-checklist-logs for best practices. You can ignore this warning if you are setting log policy per module in the Edge deployment. × production readiness: Edge Agent's storage directory is persisted on the host filesystem - Error Could not check current state of edgeAgent container × production readiness: Edge Hub's storage directory is persisted on the host filesystem - Error Could not check current state of edgeHub container √ Agent image is valid and can be pulled from upstream - OK Connectivity checks ------------------- √ container on the default network can connect to upstream AMQP port - OK × container on the default network can connect to upstream HTTPS / WebSockets port - Error Container on the default network could not connect to 192.168.1.90:443 √ container on the default network can connect to upstream MQTT port - OK √ container on the IoT Edge module network can connect to upstream AMQP port - OK × container on the IoT Edge module network can connect to upstream HTTPS / WebSockets port - Error Container on the azure-iot-edge network could not connect to 192.168.1.90:443 √ container on the IoT Edge module network can connect to upstream MQTT port - OK 27 check(s) succeeded. 3 check(s) raised warnings. Re-run with --verbose for more details. 5 check(s) raised errors. Re-run with --verbose for more details. ```

Device Information

Host OS : Debian GNU/Linux 10 (buster)
Architecture: arm64
Container OS: Linux containers

Runtime Versions

aziot-edged: iotedge 1.2.3
Edge Agent: 1.2
Edge Hub: 1.1
Docker: 20.10.5

Logs

aziot-edged logs

``` Jul 08 13:55:25 iot-gate-imx8 systemd[1]: Started Azure IoT Identity Service. Jul 08 13:55:25 iot-gate-imx8 aziot-identityd[11471]: 2021-07-08T13:55:25Z [INFO] - Starting service... Jul 08 13:55:25 iot-gate-imx8 aziot-identityd[11471]: 2021-07-08T13:55:25Z [INFO] - Version - 1.2.2 Jul 08 13:55:25 iot-gate-imx8 aziot-identityd[11471]: 2021-07-08T13:55:25Z [INFO] - Provisioning starting. Reason: Startup Jul 08 13:55:25 iot-gate-imx8 aziot-identityd[11471]: 2021-07-08T13:55:25Z [INFO] - Updated device info for red-iot-layer2-edge. Jul 08 13:55:25 iot-gate-imx8 aziot-identityd[11471]: 2021-07-08T13:55:25Z [INFO] - Provisioning complete. Jul 08 13:55:25 iot-gate-imx8 aziot-identityd[11471]: 2021-07-08T13:55:25Z [INFO] - Identity reconciliation started. Reason: Startup Jul 08 13:55:25 iot-gate-imx8 aziot-keyd[11163]: 2021-07-08T13:55:25Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"} Jul 08 13:55:25 iot-gate-imx8 aziot-keyd[11163]: 2021-07-08T13:55:25Z [INFO] - --> 200 {"content-type": "application/json"} Jul 08 13:55:25 iot-gate-imx8 aziot-keyd[11163]: 2021-07-08T13:55:25Z [INFO] - <-- POST /sign?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "398"} Jul 08 13:55:25 iot-gate-imx8 aziot-keyd[11163]: 2021-07-08T13:55:25Z [INFO] - --> 200 {"content-type": "application/json"} Jul 08 13:55:25 iot-gate-imx8 aziot-identityd[11471]: 2021-07-08T13:55:25Z [INFO] - Could not reconcile Identities with current device data. Reprovisioning. Jul 08 13:55:25 iot-gate-imx8 aziot-identityd[11471]: 2021-07-08T13:55:25Z [INFO] - Updated device info for red-iot-layer2-edge. Jul 08 13:55:25 iot-gate-imx8 aziot-keyd[11163]: 2021-07-08T13:55:25Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"} Jul 08 13:55:25 iot-gate-imx8 aziot-keyd[11163]: 2021-07-08T13:55:25Z [INFO] - --> 200 {"content-type": "application/json"} Jul 08 13:55:25 iot-gate-imx8 aziot-keyd[11163]: 2021-07-08T13:55:25Z [INFO] - <-- POST /sign?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "398"} Jul 08 13:55:25 iot-gate-imx8 aziot-keyd[11163]: 2021-07-08T13:55:25Z [INFO] - --> 200 {"content-type": "application/json"} Jul 08 13:55:25 iot-gate-imx8 aziot-identityd[11471]: 2021-07-08T13:55:25Z [ERR!] - Failed to provision with IoT Hub, and no valid device backup was found: Hub client error Jul 08 13:55:25 iot-gate-imx8 aziot-identityd[11471]: 2021-07-08T13:55:25Z [ERR!] - service encountered an error Jul 08 13:55:25 iot-gate-imx8 aziot-identityd[11471]: 2021-07-08T13:55:25Z [ERR!] - caused by: Hub client error Jul 08 13:55:25 iot-gate-imx8 aziot-identityd[11471]: 2021-07-08T13:55:25Z [ERR!] - caused by: expected value at line 1 column 1 Jul 08 13:55:25 iot-gate-imx8 aziot-identityd[11471]: 2021-07-08T13:55:25Z [ERR!] - 0: Jul 08 13:55:25 iot-gate-imx8 aziot-identityd[11471]: 1: Jul 08 13:55:25 iot-gate-imx8 systemd[1]: aziot-identityd.service: Main process exited, code=exited, status=1/FAILURE Jul 08 13:55:25 iot-gate-imx8 systemd[1]: aziot-identityd.service: Failed with result 'exit-code'. Jul 08 13:55:30 iot-gate-imx8 aziot-edged[11149]: 2021-07-08T13:55:30Z [INFO] - Obtaining edge device provisioning data... Jul 08 13:55:30 iot-gate-imx8 systemd[1]: aziot-identityd.service: Service RestartSec=5s expired, scheduling restart. Jul 08 13:55:30 iot-gate-imx8 systemd[1]: aziot-identityd.service: Scheduled restart job, restart counter is at 51. Jul 08 13:55:30 iot-gate-imx8 systemd[1]: Stopped Azure IoT Identity Service. Jul 08 13:55:30 iot-gate-imx8 aziot-edged[11149]: 2021-07-08T13:55:30Z [WARN] - The daemon could not start up successfully: Could not retrieve device information Jul 08 13:55:30 iot-gate-imx8 aziot-edged[11149]: 2021-07-08T13:55:30Z [WARN] - caused by: HTTP request error Jul 08 13:55:30 iot-gate-imx8 aziot-edged[11149]: 2021-07-08T13:55:30Z [WARN] - caused by: connection error: Connection reset by peer (os error 104) Jul 08 13:55:30 iot-gate-imx8 aziot-edged[11149]: 2021-07-08T13:55:30Z [WARN] - Requesting device reprovision. Jul 08 13:55:30 iot-gate-imx8 aziot-edged[11149]: 2021-07-08T13:55:30Z [WARN] - The reprovisioning operation failed ```

Additional Information

deployment.json on parent

Click here

```json { "modulesContent": { "$edgeAgent": { "properties.desired": { "modules": { "IoTEdgeAPIProxy": { "settings": { "image": "mcr.microsoft.com/azureiotedge-api-proxy", "createOptions": "{\"HostConfig\":{\"PortBindings\":{\"443/tcp\":[{\"HostPort\":\"443\"}]}}}" }, "type": "docker", "version": "1.0", "env": { "NGINX_DEFAULT_PORT": { "value": "443" }, "DOCKER_REQUEST_ROUTE_ADDRESS": { "value": "registry:5000" }, "BLOB_UPLOAD_ROUTE_ADDRESS": { "value": "AzureBlobStorageonIoTEdge:11002" } }, "status": "running", "restartPolicy": "always" }, "registry": { "settings": { "image": "registry:latest", "createOptions": "{\"HostConfig\":{\"PortBindings\":{\"5000/tcp\":[{\"HostPort\":\"5000\"}]}}}" }, "type": "docker", "version": "1.0", "env": { "REGISTRY_PROXY_REMOTEURL": { "value": "https://mcr.microsoft.com" } }, "status": "running", "restartPolicy": "always" }, "modbus": { "settings": { "image": "vfiotdevitem.azurecr.io/modbusmodule:0.0.2-arm64v8", "createOptions": "{\"HostConfig\":{\"Privileged\":true}}" }, "type": "docker", "version": "1.0", "status": "running", "restartPolicy": "always" } }, "runtime": { "settings": { "minDockerVersion": "v1.25", "registryCredentials": { "dev": { "address": "vfiotdevitem.azurecr.io", "password": "", "username": "" } } }, "type": "docker" }, "schemaVersion": "1.1", "systemModules": { "edgeAgent": { "settings": { "image": "mcr.microsoft.com/azureiotedge-agent:1.2", "createOptions": "" }, "type": "docker" }, "edgeHub": { "settings": { "image": "mcr.microsoft.com/azureiotedge-hub:1.1", "createOptions": "{\"HostConfig\":{\"PortBindings\":{\"5671/tcp\":[{\"HostPort\":\"5671\"}],\"8883/tcp\":[{\"HostPort\":\"8883\"}]}}}" }, "type": "docker", "status": "running", "restartPolicy": "always" } } } }, "$edgeHub": { "properties.desired": { "routes": { "route": "FROM /messages/* INTO $upstream" }, "schemaVersion": "1.1", "storeAndForwardConfiguration": { "timeToLiveSecs": 7200 } } }, "IoTEdgeAPIProxy": { "properties.desired": {} }, "registry": { "properties.desired": {} }, "modbus": { "properties.desired": {} } } } ```

Output of IoTEdgeAPIProxyModule on parent

Click here

``` 192.168.1.101 - - [09/Jul/2021:08:27:58 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:27:58 [info] 582#582: *46901 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:28:04 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:28:04 [info] 582#582: *46903 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:28:04 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:28:04 [info] 582#582: *46905 client 192.168.1.101 closed keepalive connection [2021-07-09T08:28:05Z INFO api_proxy_module::monitors::token_manager] Generating new token [2021-07-09T08:28:06Z INFO api_proxy_module::monitors::token_client] Successfully generated new token [2021-07-09T08:28:06Z INFO api_proxy_module::monitors::config_monitor] New SAS token received, reloading the config [2021-07-09T08:28:06Z INFO api_proxy_module] Request to reload Nginx received 2021/07/09 08:28:06 [notice] 583#583: signal process started 192.168.1.101 - - [09/Jul/2021:08:28:09 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:28:09 [info] 584#584: *46907 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:28:09 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:28:09 [info] 584#584: *46910 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:28:14 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:28:14 [info] 584#584: *46912 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:28:14 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:28:14 [info] 584#584: *46914 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:28:19 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:28:19 [info] 584#584: *46916 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:28:19 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:28:19 [info] 584#584: *46918 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:28:25 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:28:25 [info] 584#584: *46921 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:28:25 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:28:25 [info] 584#584: *46923 client 192.168.1.101 closed keepalive connection 2021/07/09 08:28:25 [info] 584#584: *46920 client closed connection while waiting for request, client: 192.168.1.101, server: 0.0.0.0:443 192.168.1.101 - - [09/Jul/2021:08:28:30 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:28:30 [info] 584#584: *46925 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:28:30 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:28:30 [info] 584#584: *46927 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:28:35 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:28:35 [info] 584#584: *46929 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:28:35 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:28:35 [info] 584#584: *46931 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:28:40 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:28:40 [info] 584#584: *46933 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:28:40 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:28:40 [info] 584#584: *46935 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:28:46 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:28:46 [info] 584#584: *46937 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:28:46 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:28:46 [info] 584#584: *46939 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:28:51 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:28:51 [info] 584#584: *46941 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:28:51 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:28:51 [info] 584#584: *46943 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:28:56 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:28:56 [info] 584#584: *46945 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:28:56 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:28:56 [info] 584#584: *46947 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:29:01 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:29:01 [info] 584#584: *46949 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:29:01 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:29:01 [info] 584#584: *46951 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:29:07 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:29:07 [info] 584#584: *46953 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:29:07 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:29:07 [info] 584#584: *46955 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:29:12 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:29:12 [info] 584#584: *46957 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:29:12 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:29:12 [info] 584#584: *46959 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:29:17 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:29:17 [info] 584#584: *46961 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:29:17 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:29:17 [info] 584#584: *46963 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:29:22 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:29:22 [info] 584#584: *46965 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:29:22 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:29:22 [info] 584#584: *46967 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:29:28 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:29:28 [info] 584#584: *46969 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:29:28 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:29:28 [info] 584#584: *46971 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:29:33 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:29:33 [info] 584#584: *46973 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:29:33 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:29:33 [info] 584#584: *46975 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:29:38 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:29:38 [info] 584#584: *46977 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:29:38 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:29:38 [info] 584#584: *46979 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:29:43 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:29:43 [info] 584#584: *46981 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:29:43 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:29:43 [info] 584#584: *46983 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:29:49 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:29:49 [info] 584#584: *46985 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:29:49 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:29:49 [info] 584#584: *46987 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:29:54 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:29:54 [info] 584#584: *46989 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:29:54 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:29:54 [info] 584#584: *46991 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:29:59 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:29:59 [info] 584#584: *46993 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:29:59 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:29:59 [info] 584#584: *46995 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:30:04 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:30:04 [info] 584#584: *46997 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:30:04 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:30:04 [info] 584#584: *46999 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:30:10 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:30:10 [info] 584#584: *47001 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:30:10 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:30:10 [info] 584#584: *47003 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:30:15 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:30:15 [info] 584#584: *47005 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:30:15 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:30:15 [info] 584#584: *47007 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:30:20 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:30:20 [info] 584#584: *47009 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:30:20 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:30:20 [info] 584#584: *47011 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:30:25 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:30:25 [info] 584#584: *47013 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:30:25 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:30:25 [info] 584#584: *47015 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:30:31 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:30:31 [info] 584#584: *47017 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:30:31 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:30:31 [info] 584#584: *47019 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:30:36 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:30:36 [info] 584#584: *47021 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:30:36 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:30:36 [info] 584#584: *47023 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:30:41 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:30:41 [info] 584#584: *47025 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:30:41 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:30:41 [info] 584#584: *47027 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:30:46 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:30:46 [info] 584#584: *47029 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:30:46 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:30:46 [info] 584#584: *47031 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:30:52 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:30:52 [info] 584#584: *47033 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:30:52 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:30:52 [info] 584#584: *47035 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:30:57 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:30:57 [info] 584#584: *47037 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:30:57 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:30:57 [info] 584#584: *47039 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:31:02 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:31:02 [info] 584#584: *47041 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [09/Jul/2021:08:31:02 +0000] "GET /devices/red-iot-layer2-edge/modules?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/07/09 08:31:02 [info] 584#584: *47043 client 192.168.1.101 closed keepalive connection ```

shaeussler commented 3 years ago

Hi @patrikSohlman,

check the SSL certificates: openssl s_client -connect 192.168.1.90:5671 -showcerts should return "depth=0 CN = 192.168.1.90"

CONNECTED(00000005)
depth=3 CN = Root xxxxx
verify return:1
depth=2 CN = Intermediate xxxxx
verify return:1
depth=1 CN = xxxxxx.ca
verify return:1
depth=0 CN = 192.168.1.90
verify return:1

I have done this tutorial https://docs.microsoft.com/en-us/azure/iot-edge/tutorial-nested-iot-edge?view=iotedge-2020-11 and everything works fine. In this tutorial, they add the /var/secrets/aziot/certs/red-iot-cert-test.root.ca.cert.pem certificate to the certificate store of the OS.

patrikSohlman commented 3 years ago

Hi @shaeussler I've looked at that tutorial, I did though get stuck with generating the certs on Cloud shell and decided to generate it locally. I have added the root cert to the certificate store as seen with: (I've also run sudo update-ca-certificates)

root@iot-gate-imx8:~# ls /usr/local/share/ca-certificates/
red-iot-cert-test.root.ca.cert.pem.crt

Checking the SSL certificates:

CONNECTED(00000003)
Can't use SSL_get_servername
depth=3 CN = Azure_IoT_Hub_CA_Cert_Test_Only
verify return:1
depth=2 CN = Azure_IoT_Hub_Intermediate_Cert_Test_Only
verify return:1
depth=1 CN = layer3.ca
verify return:1
depth=0 CN = 192.168.1.90
verify return:1

vadim-kovalyov commented 3 years ago

Hey @patrikSohlman, although it is hard to say for sure, it seems to me that this is connectivity problem b/w layers. The error from the service log and iotedge check made me believe that:

Jul 08 13:55:30 iot-gate-imx8 aziot-edged[11149]: 2021-07-08T13:55:30Z [WARN] - The daemon could not start up successfully: Could not retrieve device information
Jul 08 13:55:30 iot-gate-imx8 aziot-edged[11149]: 2021-07-08T13:55:30Z [WARN] -         caused by: HTTP request error
Jul 08 13:55:30 iot-gate-imx8 aziot-edged[11149]: 2021-07-08T13:55:30Z [WARN] -         caused by: connection error: Connection reset by peer (os error 104)

× container on the default network can connect to upstream HTTPS / WebSockets port - Error
    Container on the default network could not connect to 192.168.1.90:443
× container on the IoT Edge module network can connect to upstream HTTPS / WebSockets port - Error
    Container on the azure-iot-edge network could not connect to 192.168.1.90:443

Things to double check (from the docs):

Authorize outbound communications from this IP address to the parent gateway's IP address over ports 443 (HTTPS) and 5671 (AMQP). [VK: and 8883 for MQTT, just in case]

Also, in your config toml and deployment files on child device, I would change from 192.168.1.90:443/azureiotedge-agent:1.2 to $upstream:443/azureiotedge-agent:1.2 everywhere.

Hope this helps.

patrikSohlman commented 3 years ago

Hej @vadim-kovalyov, I agree with you that this seems to be the case. I ran a nmapto see if I could find anything strange. From what I gather it looks like the AMQP and MQTT port is closed, although ìotedge check states that communcation is working fine across those protocols so I'm not sure if that is what is causing the issue:

root@iot-gate-imx8:~# nmap -sT 192.168.1.90
Starting Nmap 7.70 ( https://nmap.org ) at 2021-07-12 14:54 UTC
Nmap scan report for IMX8-layer3.lan (192.168.1.90)
Host is up (0.0013s latency).
Not shown: 996 closed ports
PORT     STATE SERVICE
22/tcp   open  ssh
111/tcp  open  rpcbind
443/tcp  open  https
5000/tcp open  upnp

And to add to that, is the section of connectivity checks in ìotedge check just referencing communication in terms of messages to the edgeHub or is it all communication? I am asking just due to the fact that the check √ Agent image is valid and can be pulled from upstream - OKis passing, and that should then be looking at port 443. And the guide states that port 443 should be removed as a createOptionfor the edgeHub, and instead added as the port that the IoTEdgeAPIProxylistens to.

After the service has been running through the weekend the logs of the child device now reports:

Jul 12 14:57:15 iot-gate-imx8 aziot-edged[749]: 2021-07-12T14:57:15Z [INFO] - Checking edge runtime status
Jul 12 14:57:15 iot-gate-imx8 aziot-edged[749]: 2021-07-12T14:57:15Z [INFO] - Creating and starting edge runtime module edgeAgent
Jul 12 14:57:15 iot-gate-imx8 aziot-identityd[759]: 2021-07-12T14:57:15Z [INFO] - <-- PUT /identities/modules/$edgeAgent?api-version=2020-09-01&type=aziot {"content-type": "application/json", "host": "2f72756e2f617a696f742f6964656e74697479642e736f636b:0", "content-length": "40"}
Jul 12 14:57:15 iot-gate-imx8 aziot-keyd[764]: 2021-07-12T14:57:15Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"}
Jul 12 14:57:15 iot-gate-imx8 aziot-keyd[764]: 2021-07-12T14:57:15Z [INFO] - --> 200 {"content-type": "application/json"}
Jul 12 14:57:15 iot-gate-imx8 aziot-keyd[764]: 2021-07-12T14:57:15Z [INFO] - <-- POST /sign?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "398"}
Jul 12 14:57:15 iot-gate-imx8 aziot-keyd[764]: 2021-07-12T14:57:15Z [INFO] - --> 200 {"content-type": "application/json"}
Jul 12 14:57:15 iot-gate-imx8 aziot-identityd[759]: 2021-07-12T14:57:15Z [INFO] - !!! Hub client error
Jul 12 14:57:15 iot-gate-imx8 aziot-identityd[759]: 2021-07-12T14:57:15Z [INFO] - !!! caused by: expected value at line 1 column 1
Jul 12 14:57:15 iot-gate-imx8 aziot-identityd[759]: 2021-07-12T14:57:15Z [INFO] - --> 404 {"content-type": "application/json"}
Jul 12 14:57:15 iot-gate-imx8 aziot-edged[749]: 2021-07-12T14:57:15Z [WARN] - Error in watchdog when checking for edge runtime status:
Jul 12 14:57:15 iot-gate-imx8 aziot-edged[749]: 2021-07-12T14:57:15Z [WARN] - A module runtime error occurred.
Jul 12 14:57:15 iot-gate-imx8 aziot-edged[749]: 2021-07-12T14:57:15Z [WARN] -         caused by: HTTP response error: [404 Not Found] {"message":"Hub client error\ncaused by: expected value at line 1 column 1"}

In terms of the config and deployment files I read from the guide that I can not use $upstream/azureiotedge-agent:1.2in the config.toml as the edgeHub needs to run for that routing to work, therefor I need to hardcode it the first time it pulls the agent as stated here. The deployment.json for the child is referencing the parent with $upstream for when the agent is actually deployed the first time:

{
    "modulesContent": {
        "$edgeAgent": {
            "properties.desired": {
                "modules": {},
                "runtime": {
                    "settings": {
                        "minDockerVersion": "v1.25"
                    },
                    "type": "docker"
                },
                "schemaVersion": "1.1",
                "systemModules": {
                    "edgeAgent": {
                        "settings": {
                            "image": "$upstream:443/azureiotedge-agent:1.2",
                            "createOptions": ""
                        },
                        "type": "docker"
                    },
                    "edgeHub": {
                        "settings": {
                            "image": "$upstream:443/azureiotedge-hub:1.1",
                            "createOptions": "{\"HostConfig\":{\"PortBindings\":{\"5671/tcp\":[{\"HostPort\":\"5671\"}],\"8883/tcp\":[{\"HostPort\":\"8883\"}]}}}"
                        },
                        "type": "docker",
                        "status": "running",
                        "restartPolicy": "always"
                    }
                }
            }
        },
        "$edgeHub": {
            "properties.desired": {
                "routes": {
                    "route": "FROM /messages/* INTO $upstream"
                },
                "schemaVersion": "1.1",
                "storeAndForwardConfiguration": {
                    "timeToLiveSecs": 7200
                }
            }
        }
    }
}

vadim-kovalyov commented 2 years ago

@patrikSohlman, sorry for late reply. Do you still have connectivity issues?

About nmap, I think it does not include ports in question by default, you need to specify them in params. And you're right about $upstream, sorry for confusion.

vadim-kovalyov commented 2 years ago

I'll close this for now. Please reopen or create a new issue if you have further questions.

patrikSohlman commented 2 years ago

Hi @vadim-kovalyov, I missed your last update, sorry for that. I am still having issues, and I can´t really figure out what the next step should be. I know I had similar issues when trying this when nested IoT Edge was in preview. If I remember correctly, the fix back then was creating certificates with the hostname as CN and then adding that entry to both edgeHub and edgeAgent /etc/hosts files as well as well as the hosts /etc/hosts. I do not really like that solution as that would require me to add that at each deployment. Adding to that, the guide linked to creating certs is explicitly stating that the certs should not be named the same as the hostname: "The name passed to the create_edge_device_ca_certificate command should not be the same as the hostname parameter in the config file, or the device's ID in IoT Hub.".

I can of course try whatever you would like me to, I just need to be pointed towards the solution you think is feasible and most in line with a proper production setup (Fully aware of the fact that I need to use proper certificates for the production scenario, and not just test certs).

patrikSohlman commented 2 years ago

And please reopen if you can, I think there is value in keeping this discussion in the same issue.

huguesBouvier commented 2 years ago

Hello Patrik,

Looking back at the conversation I see an error when you try to connect using open ssl: "can't use SSL_GetServername" This will be an issue for CN checks. I also your connected status is (3) instead of (5).

Could you try again the command:openssl s_client -connect 192.168.1.90:5671 -showcerts You should see at the bottom: Can you give me the full log?

Thank you!

patrikSohlman commented 2 years ago

Yeah not a problem.

root@iot-gate-imx8:~# openssl s_client -connect 192.168.1.90:5671 -showcerts
CONNECTED(00000003)
Can't use SSL_get_servername
depth=3 CN = Azure_IoT_Hub_CA_Cert_Test_Only
verify return:1
depth=2 CN = Azure_IoT_Hub_Intermediate_Cert_Test_Only
verify return:1
depth=1 CN = layer3.ca
verify return:1
depth=0 CN = 192.168.1.90
verify return:1
---
Certificate chain
 0 s:CN = 192.168.1.90
   i:CN = layer3.ca
-----BEGIN CERTIFICATE-----
MIID1zCCAb+gAwIBAgIBADANBgkqhkiG9w0BAQsFADAUMRIwEAYDVQQDDAlsYXll
cjMuY2EwHhcNMjEwNzE0MTMyNjE1WhcNMjEwODEzMTMyNjE1WjAXMRUwEwYDVQQD
DAwxOTIuMTY4LjEuOTAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDh
zYTC+K6XOVdEOeqpZtLSBH4yHEXpn74fCNxS8uHfxdzcjW5YH+bviSzcxszcq893
xOxn4NX/nk5xTwSXzH7CDLNVFok05/honQvAUmnhwXjWAK71ganjX628IfUwtCK8
7GsRgYe8CNwzZ6Xcc6yOqEgQa99lQKoyt8mr3JsMIOzZfi/3yaADMfcXvOO3bmAV
+XEMgw1vhtY8jKvynF8pHkhItyrGGjcsmVJodbCJpXGrqeP6BvBrekHkRsn+qybc
Z8k+PUMmh/FV+ZdK3JuaagHHNg0KQ5RNBlLxRNt1MT8/yFk7OpPxiVXqdzP7ZXW7
mFCCWkPQCWSbHRUHtLaTAgMBAAGjMTAvMBMGA1UdJQQMMAoGCCsGAQUFBwMBMBgG
A1UdEQQRMA+CB2VkZ2VodWKHBMCoAVowDQYJKoZIhvcNAQELBQADggIBAF8BOsf8
6w4fOFVTO24IaM2fkEsYu28KXJ7XLAFdPAYuSZ8TlzDf3NNy4rNdt0YNa/696YxX
/Ef6prj8eS4FkGZAePj39nZizs5G8EdHubHwFB00FoTRnrOK4OMpgbxceBWR+Igy
B2MOwFvCNFU4clGx2ccltkInTa/Bws9K4tcfhrKQAGHzc1iR5jAhtz29SjB9LhGt
dl7kIUUIN66X+hPH86NRoTk2aDl4AZcFj2Ovp/rIndszoUR9YFyGXn4lbLDcAyjd
N70xDPgEoNayxazfuunxiwqqoBMAxF38QHPbgpkUBL41/o0MAtHhmWujbw11HPFX
2w/mAD/fapgnpgEqxqMD5UeSmi0m9NHPfs068uQO8m4Ud88Mcdq3Jv48zdhv/g93
8m4QF3bs53bLgd/UaxLkD9jnl4JAcMdvByXUKMsebZu5rvkORgquP2TKEtlgVo4Y
yOi1C0qnkrJEPFG/RYnrTpRyIIt2xO6+Jv3gs0618FZFL2Jp8J1TIw1kRMELzqrx
DPms9r1D+fisS09np6Be5HVhrpJ50l8lJkxsNHYlOoGAKj3ELO5hUe6ig3Cg8O7s
MJOqo/aTs1ApRJ5nW55aRbol2TGuDMvVyP7IfEPL/9O8UjdoqoTjZt7+/Vc2WD1y
dpCbnash+pVsfZsuefbCfrjmD+a6BDmZTuVc
-----END CERTIFICATE-----
 1 s:CN = layer3.ca
   i:CN = Azure_IoT_Hub_Intermediate_Cert_Test_Only
-----BEGIN CERTIFICATE-----
MIIFXTCCA0WgAwIBAgICEAYwDQYJKoZIhvcNAQELBQAwNDEyMDAGA1UEAwwpQXp1
cmVfSW9UX0h1Yl9JbnRlcm1lZGlhdGVfQ2VydF9UZXN0X09ubHkwHhcNMjEwNzA4
MTI0OTA0WhcNMjMwNjA4MTI0OTA0WjAUMRIwEAYDVQQDDAlsYXllcjMuY2EwggIi
MA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQDdj7g6CXkqMGlhu+1nNvQfa6cc
CDrSpUJ/RRS3hsMF9ao+mhIPSZYTMnHy+wg28+dsfXxgJCoFdrlhEclMPaXFjBW8
vXZcc4DSnHmNTifNzbYHp0HiEnSXcMzj+3/lG4W7yTeDv4tcmZS8RtRIlyKDSA5u
QATzFVMG/CpHEN+exwKoMA1ZqKhv0VdBDDbmJJqdcGzoOSzSuBEDAvO114p/Cgr0
OTbVr/9unhlU3sGWyxR8KDbyN7OtUpZSEnU1JOgmhN1sdHtNWnKqGJyHDzasyWue
7XQW8A/LJuyxliq+BWt5S5X/Z+QlrvKMSQJ1uYhElKIEG8wabWROKqs1GAz6D/CV
cdGqY9iE0Dj2f3RJNE4qdD4jKkdjXJP1nVPvAjhGXm7ExmTz53ZKNoD0+kOxxaub
lEobqVrrPtkGFnlTjC0nCVzZ9uZaOf3tQROwAjCZ2oZcqhxoX2NHep4dGwxEGUl6
DnykIK8KbZzx8Kx47IFSTSqxB2w+vGgMke7jQkcqwzyRjU78LY8gOIk2k6j2Vvuk
8C+BM96sJPmcWFfemvoD+x9vxXv1QZRzuV6cHkgRy6enpLFZma0/hK64wdlHvNPW
FXvSRIbLQt5Ky3cD9oSCIHIT2WZX3QxuV4zxgCjU1yHL6QDNXzjxKm9TGwxZb3Gi
zXCvVVf6IgGxN51j0QIDAQABo4GYMIGVMB0GA1UdDgQWBBStxBKEzHQUjnrHcUOc
+WVlZI+z3zBTBgNVHSMETDBKgBQQ7PT6K2MVBuYD1NlwR1R1KiQg66EupCwwKjEo
MCYGA1UEAwwfQXp1cmVfSW9UX0h1Yl9DQV9DZXJ0X1Rlc3RfT25seYICEAAwDwYD
VR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAYYwDQYJKoZIhvcNAQELBQADggIB
AFkiCGKPQ/SQwRo1e3hL0iFU4OZiRLKu0pm/9off6/1DNs8N7VbQ7iIyJx+mpWzA
S/I5F3bjPpOv5oRrDTOOgBzmiToJ31cBSx2T6XT1a5o/3OOBR/YiwC2mo7R1A4Z3
UXyMJPPJJQpeCVRW9EZq+iJHIOVuj5+o4ffwIdBD9nWKO+jlC5XRhSXkeIVgTG6m
r4VM8WvLRrpTeyHRh7z4wBqwNcKctU4xPsDQiSJSg45zaQCdgn9IRWdIyoh5qVB3
XwAfwUZ+jbktNC4AQbjr7MkpVSrtbCilxLmCdACQAHfLQb9z8BOzqCp66QCKXz8j
3Fke0kdV7k1L3rt14yKttdqM2q5mTxt5sv/2fq7HuiyFQKqBbz8WklSKH745KN6I
fSjPjCsRsur9fKPLNFuKW1CIuDMQMLSFM6OrgcRI9audXqNOeAyMhrl/EaVhMXOb
jniOnz1Jdzfs4Cv7S5OOrFbvzKBJArwbdiUSY4kAa7Pw8BLkz6FbdBwsWHbUy2yI
JCWbDgMTprBPwufIszizG2r58yQuEASuZd9i3qzfTeFrkfJv+P4mxO8p/uO/6Jbc
aXM0tCJ58xq4eG8WetIILFOfnFljDegwgKurlmiBNL9LQSGW3mSmHXkhQJREa+8L
EKAWWNpPS6QOUXh4NfDINAwRUUttiwGDJhlNxkxtI8x+
-----END CERTIFICATE-----
 2 s:CN = Azure_IoT_Hub_Intermediate_Cert_Test_Only
   i:CN = Azure_IoT_Hub_CA_Cert_Test_Only
-----BEGIN CERTIFICATE-----
MIIFhTCCA22gAwIBAgICEAAwDQYJKoZIhvcNAQELBQAwKjEoMCYGA1UEAwwfQXp1
cmVfSW9UX0h1Yl9DQV9DZXJ0X1Rlc3RfT25seTAeFw0yMTA2MjgxMTU2NTJaFw0y
MzA1MjkxMTU2NTJaMDQxMjAwBgNVBAMMKUF6dXJlX0lvVF9IdWJfSW50ZXJtZWRp
YXRlX0NlcnRfVGVzdF9Pbmx5MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKC
AgEAwS+oR6V7zLIBwnnsJKv9iifvKt6AdVSqoK54zy2gn7x1QBlitfXnfumir+qf
LtCJhObsrMM0GgHetVturgvASLU7l2+qlzYmykkZkjZ/OV4CL0wzdsWh2Dk/f74z
Ko2ngE264o30k10YqUVME5dxxxNdx7kLFA4gRN0rZj4FHBSH8+aSkV8wzG5u4X0O
lJMB08+rc2TkIJ+QNCoNjqCZ10I8QqI913ZA/kpooEfFgkor172jQ1eV9LXwezPm
pvtaHUssvU4pcPCPFgr80KnLENsPEe/LtyFH53oF8qOD5BWvD6fH/qZuyWH7j4FN
JgQKUnw2IPP+XPL5mgpGa3DVD87fVyeUXVYwRpnuoS85UoZjbB9gB0xgIoiBKt4P
KziVpu+3OGHIFJxgmhhH6cBTW7b10yCZKYEVyLLKeOJ1IPFB+piwSY8U9FPYsj8V
Z262640SKLGuVNwDekU3wZjSVOe0TADWlbvooc29m+eY2f7S5Sr05f0ACJoLD79C
VD6HRsRf6WaUY18xZ/NHAHiMSvBM3uj3eNZQgqMenhdTT990pUIKPstosnNlr5PK
GkOQKCRVbeODuyWakDB73MLTHdFnz1CPHabbNqwxLksv2xiivnB8sVxtcXwYfAQK
JtX5PuNdrgccJ8yvXgQuA3okXJtsgeviK+Ftg/5GsjinEmkCAwEAAaOBqjCBpzAd
BgNVHQ4EFgQUEOz0+itjFQbmA9TZcEdUdSokIOswZQYDVR0jBF4wXIAUPIOl86r3
yBhmZl/yt+eidYSyftihLqQsMCoxKDAmBgNVBAMMH0F6dXJlX0lvVF9IdWJfQ0Ff
Q2VydF9UZXN0X09ubHmCFB7dR3ctFf+w9pbN4xjonDhgjn5lMA8GA1UdEwEB/wQF
MAMBAf8wDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4ICAQBOqPtYej7M
pfnTFFDo8DMEXdosgIg2sz92VIqJPtgRKR5qkM/0fL+6n5iGJ2Gmk82872nEzWi1
BzFkMB6CUX1yW9NlC9r0QTfAhEiye2rthCig4sdw7KPi8gcXl/SWNLWfFaXU/JPb
LqiRIcuKmVjCXQ6aY4bAGp0BHISzw4endYpNK0IxOESuONGz2pMWZ0PgdB1BqBuB
jMlfxU0jQbAz8NAJIqsVegEWPf6TOrK9vkhL7q0b21poCD5/7SFJjJbzb3DflcLU
KQ4vVMVB0/YqDBfc8a/qOxufbOWOlOwK3Wiq8jr0HTYKR/+vU9ZIk+ayJr9ToTFT
xVYgchOSPmyLsIPybyU4ZVCXnijayMsq39bDR7GP/hckHWv3VWD22Rlp4Up0sCsb
WCi6+jlkKWH3fenF7672wmaYGCCA2LM5etQj4sdv48mMA0jS4dQVKG2sZW6gUUkS
5uzy5V8v/KE7yxLQTkTzZ6aWzkDaMRA2SAKkPBFTxozb4adD48Eiqiy88iOCKXTA
HplkwGEAneiPm6sj/AyQv2K3uczsBLUH0eWdtpZllUrpBXX0hSAjppDeeozFaQYy
eBcgQPzUNx/MyczMPaFLWNyeWJA+P5Zmt6ezfAitQZwHKX9DE09rrMKgFOcXPn66
zo9esZChhleakSHMBx8k9O9/mN0JeHn/LQ==
-----END CERTIFICATE-----
---
Server certificate
subject=CN = 192.168.1.90

issuer=CN = layer3.ca

---
No client certificate CA names sent
Client Certificate Types: RSA sign, DSA sign, ECDSA sign
Requested Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:ECDSA+SHA1:RSA+SHA224:RSA+SHA1:DSA+SHA224:DSA+SHA1:DSA+SHA256:DSA+SHA384:DSA+SHA512
Shared Requested Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:RSA+SHA224:DSA+SHA224:DSA+SHA256:DSA+SHA384:DSA+SHA512
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 4475 bytes and written 388 bytes
Verification: OK
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: 0E3F50F3EEDDBD0C174937BF25FDF2C4DCAC4536241C2E7C893BEF58AC2B3F98
    Session-ID-ctx:
    Master-Key: 0A1E2361A5D48B5BD9E6F5418DD7E9D6EAD3E74ABD8E64DCA5990E5F905D0D030C07397DDA1798593AE67AD6E0EC5C71
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 7200 (seconds)
    TLS session ticket:
    0000 - 0f 0a 5d f5 b2 dd f0 87-7c db ef 54 ed e1 8c 46   ..].....|..T...F
    0010 - 40 fc d5 82 e5 87 e6 8f-54 db 19 2c f5 4b 73 a9   @.......T..,.Ks.
    0020 - 87 a3 6d 0a 87 2a 3e 5b-a5 a5 c6 ab 3b d2 3f 7b   ..m..*>[....;.?{
    0030 - f0 1e e6 af d4 88 98 29-b8 05 57 d8 ea 7d 7b 6e   .......)..W..}{n
    0040 - 16 06 26 8e 63 a8 78 df-1b 07 94 26 82 ca 10 ab   ..&.c.x....&....
    0050 - ad e1 a1 74 61 6a fe e6-ff 80 56 c7 ae b6 bc af   ...taj....V.....
    0060 - 21 0e 47 58 be f6 bf b6-a7 ce f8 18 fc 1d 29 36   !.GX..........)6
    0070 - f4 9f 47 db a0 7d 0e 89-a7 b5 f5 5d 72 89 25 40   ..G..}.....]r.%@
    0080 - cb 43 2a d8 00 ed 9a 1b-b5 ff 6f 4f f2 04 f5 d6   .C*.......oO....
    0090 - 46 ab d9 c0 4d 2b c7 51-0c 2a a7 1a a0 b0 1c bb   F...M+.Q.*......

    Start Time: 1627719184
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: yes
---

I am thinking that this is the underlying issue, and as stated above, the fix might be to name the certs the same as the hostnames and utilizing /etc/hosts for proper routing, thankful for any other suggestions of course.

melsayed1981 commented 2 years ago

we are having the exact same problem. what I am thinking it is an open_ssl issue

nav671 commented 2 years ago

I am having the same issue as well in my Nested Edge Gateway Scenario. My Gateway Device works fine and all necessary modules are deployed (Agent, Hub, APIProxy, costum registry with necessary modules for both devices). My downstream device is also an IoT Edge device with the necessary x509 certificates from the certificate hierarchy of the test-only azure iot root certificate. When I use:

 openssl s_client -connect 192.168.178.77:443 -showcerts

I also get:

CONNECTED(00000003)
Can't use SSL_get_servername
depth=3 CN = Azure_IoT_Hub_CA_Cert_Test_Only
verify return:1
depth=2 CN = Azure_IoT_Hub_Intermediate_Cert_Test_Only
verify return:1
depth=1 CN = GATEWAY.ca
verify return:1
depth=0 CN = 192.168.178.77
verify return:1

with a return code: 0 in the end

Start Time: 1629216385
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: yes

So the connection to my Gateway device is possible and working? Or is the error with "can't use SSL_get_servername" the reason for the errors?

For iotedge check --verbose I get the following error, which is repeated in the connectivity checks:

× configuration has correct URIs for daemon mgmt endpoint - Error
    Unable to find image '192.168.178.77/azureiotedge-diagnostics:1.2.3' locally
    docker: Error response from daemon: manifest for 192.168.178.77/azureiotedge-diagnostics:1.2.3 not found: manifest unknown: manifest unknown.
    See 'docker run --help'.
        caused by: docker returned exit code: 125, stderr = Unable to find image '192.168.178.77/azureiotedge-diagnostics:1.2.3' locally
                   docker: Error response from daemon: manifest for 192.168.178.77/azureiotedge-diagnostics:1.2.3 not found: manifest unknown: manifest unknown.
                   See 'docker run --help'.

I also get the same errors as @patrikSohlman for the iotedge system logs.

Error in watchdog when checking for edge runtime status: A module runtime error occurred.
caused by: HTTP response error:
[404 Not Found] {"message":"Hub client error\ncaused by: expected value at line 1 column 1"}

huguesBouvier commented 2 years ago

@nav671 The error you are seeing for IoTedge check is probably because the API proxy is listening on port 8000 (this is the default). Then you need to do: iotedge checks --diagnostics-image-name 192.168.178.77:8000/azureiotedge-diagnostics:1.2.3.

I am not sure about the "Can't use SSL_get_servername" but if it can connect successfully ( Verify return code: 0 (ok)) it is not a problem.

huguesBouvier commented 2 years ago

@patrikSohlman It seems that the issue happens when child edge is trying to connect edgeHub through port 443. I have looked at the API proxy logs, however I could not see any evidence in it.

Could you do:

On the child, openssl s_client -connect 192.168.1.90:443 -showcerts => I see that iotedge edge check is having trouble on port 443, this may give me more info.
On the child, sudo iotedge check --verbose. Also to get more info why it doesn't connect on 443.
The full API proxy logs
On the parent, the full edgeHub logs.

Hopefully this will tell me where the message to create identity is getting blocked.

patrikSohlman commented 2 years ago

@huguesBouvier Sure, not a problem. 1.

Click here

``` openssl s_client -connect 192.168.1.90:443 -showcerts CONNECTED(00000003) Can't use SSL_get_servername depth=3 CN = Azure_IoT_Hub_CA_Cert_Test_Only verify return:1 depth=2 CN = Azure_IoT_Hub_Intermediate_Cert_Test_Only verify return:1 depth=1 CN = layer3.ca verify return:1 depth=0 CN = 192.168.1.90 verify return:1 --- Certificate chain 0 s:CN = 192.168.1.90 i:CN = layer3.ca -----BEGIN CERTIFICATE----- MIID3zCCAcegAwIBAgIBADANBgkqhkiG9w0BAQsFADAUMRIwEAYDVQQDDAlsYXll cjMuY2EwHhcNMjEwODI0MDkzNTEzWhcNMjEwOTIzMDkzNTEzWjAXMRUwEwYDVQQD DAwxOTIuMTY4LjEuOTAwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDy KssUT/GP3bBu+9s3c1pscdyZmXig6Q/8iVKQzHXDGaP/1Yzv15AFliWUpyfUqVQo c/RLEiHFakqZNd6OJS9KnX54CsHhd9IOkhSj95vmVTVlrg8HTXAryVlGwfqkIU96 FCd7zooCTQvMmCuNG1y2QSSMZtWcH1BZWvuDQaPQxgERlp7jZFZfQtH0EOb6WpnN leQ/FvM9OkNZLWd7ZpQ/R+nOtIOMMC6zC+cpLhhTwMYvJuwjFzOzojeXRNOTeWOF 5/K6n/yc+RvoeoT4Zte2HGEt7Z3zVcsuNxFUEJ8cdWCHRiQLTrsFXXTrhzQlmf5r t9DJRkwI6BqzFDgCR4z1AgMBAAGjOTA3MBMGA1UdJQQMMAoGCCsGAQUFBwMBMCAG A1UdEQQZMBeCD2lvdGVkZ2VhcGlwcm94eYcEwKgBWjANBgkqhkiG9w0BAQsFAAOC AgEAH4k0DQ1o8G0ciK4jHOn81gXa2myMU/LOQrxWAjMa5BwfRSInILRzjPIOMR2y JFYhDFZZ9HBTslQphG3/3/U61JkTPxmYJEFSTeuW34DSXP0rDo/5FxGc6crIL+gG yJ3/7D33cm5CTeGRwhBA3cKuXsnxsDI5TaPj/zB9f01MjvRhBfUIk/znxzYdapP3 Gs7KBYcaP9Ohqbjue+8ZhRW1T4OacYK+a//EEgH80Y8JYjs+HCPEfH+zwegNPIE5 pnbnjzZNELnG72sr0LriiO8Yet1GASfdC+cpEacZPeb7HHnMKX6/TZCuxlzcMpdX mq9brBhF2iH+/R0sSFzwBedXVxQJ7nlP2UgegVum2N+4Ro+LrXY/3XVgp6m7czc0 9GkL9r4mTaJ2YMbCViJAwUnsPsEhhQKm2CglHomWBFeBLVDcQcjeORActSTnOuyO j2DiUAqvjDti5lNjTX6tKdcSMAfVJ0TCgF7RqkfgUal7qc1nDqnV5PY6iblxkgVS Gp5ZC6uq8X/SwWyZrF0FRs8XQQsY78+h4gPjdk02R+vK05V5nhFwJ9T3+3Hsj5bD f75rIA9akn6DSZuRyQ/V3rKTg5FWZSvZrmyOipjLL6W03rk7eykcas6TLe+K6zYA h4ukQ/4zAcJi8asi6m8ixYIIamKR+lk7o9GeaG0qB+j+gH8= -----END CERTIFICATE----- 1 s:CN = layer3.ca i:CN = Azure_IoT_Hub_Intermediate_Cert_Test_Only -----BEGIN CERTIFICATE----- MIIFXTCCA0WgAwIBAgICEAYwDQYJKoZIhvcNAQELBQAwNDEyMDAGA1UEAwwpQXp1 cmVfSW9UX0h1Yl9JbnRlcm1lZGlhdGVfQ2VydF9UZXN0X09ubHkwHhcNMjEwNzA4 MTI0OTA0WhcNMjMwNjA4MTI0OTA0WjAUMRIwEAYDVQQDDAlsYXllcjMuY2EwggIi MA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQDdj7g6CXkqMGlhu+1nNvQfa6cc CDrSpUJ/RRS3hsMF9ao+mhIPSZYTMnHy+wg28+dsfXxgJCoFdrlhEclMPaXFjBW8 vXZcc4DSnHmNTifNzbYHp0HiEnSXcMzj+3/lG4W7yTeDv4tcmZS8RtRIlyKDSA5u QATzFVMG/CpHEN+exwKoMA1ZqKhv0VdBDDbmJJqdcGzoOSzSuBEDAvO114p/Cgr0 OTbVr/9unhlU3sGWyxR8KDbyN7OtUpZSEnU1JOgmhN1sdHtNWnKqGJyHDzasyWue 7XQW8A/LJuyxliq+BWt5S5X/Z+QlrvKMSQJ1uYhElKIEG8wabWROKqs1GAz6D/CV cdGqY9iE0Dj2f3RJNE4qdD4jKkdjXJP1nVPvAjhGXm7ExmTz53ZKNoD0+kOxxaub lEobqVrrPtkGFnlTjC0nCVzZ9uZaOf3tQROwAjCZ2oZcqhxoX2NHep4dGwxEGUl6 DnykIK8KbZzx8Kx47IFSTSqxB2w+vGgMke7jQkcqwzyRjU78LY8gOIk2k6j2Vvuk 8C+BM96sJPmcWFfemvoD+x9vxXv1QZRzuV6cHkgRy6enpLFZma0/hK64wdlHvNPW FXvSRIbLQt5Ky3cD9oSCIHIT2WZX3QxuV4zxgCjU1yHL6QDNXzjxKm9TGwxZb3Gi zXCvVVf6IgGxN51j0QIDAQABo4GYMIGVMB0GA1UdDgQWBBStxBKEzHQUjnrHcUOc +WVlZI+z3zBTBgNVHSMETDBKgBQQ7PT6K2MVBuYD1NlwR1R1KiQg66EupCwwKjEo MCYGA1UEAwwfQXp1cmVfSW9UX0h1Yl9DQV9DZXJ0X1Rlc3RfT25seYICEAAwDwYD VR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAYYwDQYJKoZIhvcNAQELBQADggIB AFkiCGKPQ/SQwRo1e3hL0iFU4OZiRLKu0pm/9off6/1DNs8N7VbQ7iIyJx+mpWzA S/I5F3bjPpOv5oRrDTOOgBzmiToJ31cBSx2T6XT1a5o/3OOBR/YiwC2mo7R1A4Z3 UXyMJPPJJQpeCVRW9EZq+iJHIOVuj5+o4ffwIdBD9nWKO+jlC5XRhSXkeIVgTG6m r4VM8WvLRrpTeyHRh7z4wBqwNcKctU4xPsDQiSJSg45zaQCdgn9IRWdIyoh5qVB3 XwAfwUZ+jbktNC4AQbjr7MkpVSrtbCilxLmCdACQAHfLQb9z8BOzqCp66QCKXz8j 3Fke0kdV7k1L3rt14yKttdqM2q5mTxt5sv/2fq7HuiyFQKqBbz8WklSKH745KN6I fSjPjCsRsur9fKPLNFuKW1CIuDMQMLSFM6OrgcRI9audXqNOeAyMhrl/EaVhMXOb jniOnz1Jdzfs4Cv7S5OOrFbvzKBJArwbdiUSY4kAa7Pw8BLkz6FbdBwsWHbUy2yI JCWbDgMTprBPwufIszizG2r58yQuEASuZd9i3qzfTeFrkfJv+P4mxO8p/uO/6Jbc aXM0tCJ58xq4eG8WetIILFOfnFljDegwgKurlmiBNL9LQSGW3mSmHXkhQJREa+8L EKAWWNpPS6QOUXh4NfDINAwRUUttiwGDJhlNxkxtI8x+ -----END CERTIFICATE----- 2 s:CN = Azure_IoT_Hub_Intermediate_Cert_Test_Only i:CN = Azure_IoT_Hub_CA_Cert_Test_Only -----BEGIN CERTIFICATE----- MIIFhTCCA22gAwIBAgICEAAwDQYJKoZIhvcNAQELBQAwKjEoMCYGA1UEAwwfQXp1 cmVfSW9UX0h1Yl9DQV9DZXJ0X1Rlc3RfT25seTAeFw0yMTA2MjgxMTU2NTJaFw0y MzA1MjkxMTU2NTJaMDQxMjAwBgNVBAMMKUF6dXJlX0lvVF9IdWJfSW50ZXJtZWRp YXRlX0NlcnRfVGVzdF9Pbmx5MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKC AgEAwS+oR6V7zLIBwnnsJKv9iifvKt6AdVSqoK54zy2gn7x1QBlitfXnfumir+qf LtCJhObsrMM0GgHetVturgvASLU7l2+qlzYmykkZkjZ/OV4CL0wzdsWh2Dk/f74z Ko2ngE264o30k10YqUVME5dxxxNdx7kLFA4gRN0rZj4FHBSH8+aSkV8wzG5u4X0O lJMB08+rc2TkIJ+QNCoNjqCZ10I8QqI913ZA/kpooEfFgkor172jQ1eV9LXwezPm pvtaHUssvU4pcPCPFgr80KnLENsPEe/LtyFH53oF8qOD5BWvD6fH/qZuyWH7j4FN JgQKUnw2IPP+XPL5mgpGa3DVD87fVyeUXVYwRpnuoS85UoZjbB9gB0xgIoiBKt4P KziVpu+3OGHIFJxgmhhH6cBTW7b10yCZKYEVyLLKeOJ1IPFB+piwSY8U9FPYsj8V Z262640SKLGuVNwDekU3wZjSVOe0TADWlbvooc29m+eY2f7S5Sr05f0ACJoLD79C VD6HRsRf6WaUY18xZ/NHAHiMSvBM3uj3eNZQgqMenhdTT990pUIKPstosnNlr5PK GkOQKCRVbeODuyWakDB73MLTHdFnz1CPHabbNqwxLksv2xiivnB8sVxtcXwYfAQK JtX5PuNdrgccJ8yvXgQuA3okXJtsgeviK+Ftg/5GsjinEmkCAwEAAaOBqjCBpzAd BgNVHQ4EFgQUEOz0+itjFQbmA9TZcEdUdSokIOswZQYDVR0jBF4wXIAUPIOl86r3 yBhmZl/yt+eidYSyftihLqQsMCoxKDAmBgNVBAMMH0F6dXJlX0lvVF9IdWJfQ0Ff Q2VydF9UZXN0X09ubHmCFB7dR3ctFf+w9pbN4xjonDhgjn5lMA8GA1UdEwEB/wQF MAMBAf8wDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4ICAQBOqPtYej7M pfnTFFDo8DMEXdosgIg2sz92VIqJPtgRKR5qkM/0fL+6n5iGJ2Gmk82872nEzWi1 BzFkMB6CUX1yW9NlC9r0QTfAhEiye2rthCig4sdw7KPi8gcXl/SWNLWfFaXU/JPb LqiRIcuKmVjCXQ6aY4bAGp0BHISzw4endYpNK0IxOESuONGz2pMWZ0PgdB1BqBuB jMlfxU0jQbAz8NAJIqsVegEWPf6TOrK9vkhL7q0b21poCD5/7SFJjJbzb3DflcLU KQ4vVMVB0/YqDBfc8a/qOxufbOWOlOwK3Wiq8jr0HTYKR/+vU9ZIk+ayJr9ToTFT xVYgchOSPmyLsIPybyU4ZVCXnijayMsq39bDR7GP/hckHWv3VWD22Rlp4Up0sCsb WCi6+jlkKWH3fenF7672wmaYGCCA2LM5etQj4sdv48mMA0jS4dQVKG2sZW6gUUkS 5uzy5V8v/KE7yxLQTkTzZ6aWzkDaMRA2SAKkPBFTxozb4adD48Eiqiy88iOCKXTA HplkwGEAneiPm6sj/AyQv2K3uczsBLUH0eWdtpZllUrpBXX0hSAjppDeeozFaQYy eBcgQPzUNx/MyczMPaFLWNyeWJA+P5Zmt6ezfAitQZwHKX9DE09rrMKgFOcXPn66 zo9esZChhleakSHMBx8k9O9/mN0JeHn/LQ== -----END CERTIFICATE----- 3 s:CN = Azure_IoT_Hub_CA_Cert_Test_Only i:CN = Azure_IoT_Hub_CA_Cert_Test_Only -----BEGIN CERTIFICATE----- MIIFjTCCA3WgAwIBAgIUHt1Hdy0V/7D2ls3jGOicOGCOfmUwDQYJKoZIhvcNAQEL BQAwKjEoMCYGA1UEAwwfQXp1cmVfSW9UX0h1Yl9DQV9DZXJ0X1Rlc3RfT25seTAe Fw0yMTA2MjgxMTU2NTBaFw0yMzA1MjkxMTU2NTBaMCoxKDAmBgNVBAMMH0F6dXJl X0lvVF9IdWJfQ0FfQ2VydF9UZXN0X09ubHkwggIiMA0GCSqGSIb3DQEBAQUAA4IC DwAwggIKAoICAQDpdbdNWAYl1Ehbxfk87E8ul+UHm+/1lcK600JEx5iEYjzWTQZ7 hS/GEhP1sMZUb9FuHcrBQs1kxFSHb1XGDqoYuJ31SpwzyLILbCP6K1fFoYrDsO4Q tk88ftLzFIUiZHjKhX0Gt9FPMmLiosROqzDraUi016JDNGS094mfPWDmH6KLfb2I jQRp4f5Zww0OUcaZ8AAfNwyzTle8CCQ45f1qNXVszQeM7AB055YgsiYLIUBr26xP Co3LH/preSgjMjiCApTiTRBg0BywvFX6bKcXG4H19o+06DHQMTHGU9EDKANheIEK 4C0jr702sH21zlW/0hTEcHOc+0zkpH0IdgQhvr4EmlmEjAN5S+KnluH4uPkZRZg0 MggK6UFBxQe/og7sm5SDyM+8YttUxqPOPhLGieNQNPmcrI+zBQdso5Gc2YzG/m2K +UOnZgpGBNE6ivb3pTffMCaecLdiIr/iJmyg68lOLhhyQtjLGugpwZdNll8i0fhJ 5vNEZXYkRXgbwskhd6kX0Yym65LNW8Y5wAAvpUcsOuhtec/uLhpM1Vcpah049Hd0 rxn9vvpovMELhPwiERi7MEA4mZ5ZrXqsZmHFfmpdyaT9xLoASQpefJEUVhk0kOzt eUY0fDksBGA67bjnMmoXP3dOGu518irRj4VGsfdELivwSsNys1F0EBF8TwIDAQAB o4GqMIGnMB0GA1UdDgQWBBQ8g6XzqvfIGGZmX/K356J1hLJ+2DBlBgNVHSMEXjBc gBQ8g6XzqvfIGGZmX/K356J1hLJ+2KEupCwwKjEoMCYGA1UEAwwfQXp1cmVfSW9U X0h1Yl9DQV9DZXJ0X1Rlc3RfT25seYIUHt1Hdy0V/7D2ls3jGOicOGCOfmUwDwYD VR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAYYwDQYJKoZIhvcNAQELBQADggIB AG7PkjpAnBUH3ppBG7jV2/ZCFVvBuwQQqUH02YmtTxKXkU7jc0EzMuP6/t4WSz5u l+cIYCpQMaG6dVrL5vRaucKe+7omlOoiwDVGZNKoO7rMWYJsYtwM/5UIqBhqGLAM XScf2tYJGfGFN++lA/ctiF6AVdYJQv8HBpvw1Ai8aBRD5Ej2ZJ3nuW9P173slbCK dq3vL6cpU3/DVIb0EC93K+tK72mHPvwv/gSh1twS7lD/rWzFH+7sLjKQ5n0rkkoi 6g2qGJF9DOASUs5FEwtT/82ffceH7S0AgnkhbF4GBfiLAq0TkEkftZLH3Na0REtg UswQoZV9Tt46SM6OcRYgxPGJ4QnQXzmfPOixewW4VrwIasde9k4IJ0R+i0RCmVPk yyI3sw4E4zl3ECnpPKpmL45n/EFzMYjeKsCZVsMSS1MM4kN4NCviFKsBdjeHjTpQ lPUSQYMH1m/2gzrWRIcp2FTLsRxn05DuyDBarkozGZSbenwQd1OJM/98iw/50vw/ 9JBe61PFGV5x7GdGz9QbHdmyuXaZGpSw1CymQmLZlNx0i7lIrTmlRoiTikXSEO/1 yjWrMsPajWxMSTVDf3UlOOsXxMoVeLfnL7jSjQu0p77IdII1tCB/uhy8qFR1OuRT 1/ecJBsR+9M1jRtFgPMROaz7uzWyRB8Ek1OcyeVb46kN -----END CERTIFICATE----- --- Server certificate subject=CN = 192.168.1.90 issuer=CN = layer3.ca --- Acceptable client certificate CA names CN = layer3.ca CN = Azure_IoT_Hub_Intermediate_Cert_Test_Only CN = Azure_IoT_Hub_CA_Cert_Test_Only Client Certificate Types: RSA sign, DSA sign, ECDSA sign Requested Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:RSA+SHA224:DSA+SHA224:DSA+SHA256:DSA+SHA384:DSA+SHA512 Shared Requested Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:RSA+SHA224:DSA+SHA224:DSA+SHA256:DSA+SHA384:DSA+SHA512 Peer signing digest: SHA256 Peer signature type: RSA-PSS Server Temp Key: X25519, 253 bits --- SSL handshake has read 6047 bytes and written 388 bytes Verification: OK --- New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384 Server public key is 2048 bit Secure Renegotiation IS supported Compression: NONE Expansion: NONE No ALPN negotiated SSL-Session: Protocol : TLSv1.2 Cipher : ECDHE-RSA-AES256-GCM-SHA384 Session-ID: 22B37F9475D8C2D96E61867E43278B5860F3935006E37B7A964D19B3479BD6EC Session-ID-ctx: Master-Key: B7A6C7DF695C984F99468CBED80221CA9AB697EF759FE4BD2A940F38C143ED6BC1C6D2DD0E401A9AB798701CA751312C PSK identity: None PSK identity hint: None SRP username: None TLS session ticket lifetime hint: 300 (seconds) TLS session ticket: 0000 - e4 06 0e 14 28 0a 04 95-9f e5 ba d4 9c 5a 02 4e ....(........Z.N 0010 - ca 02 7b b3 0a b1 ea e4-0f c1 2f c2 a6 09 56 b3 ..{......./...V. 0020 - 63 24 34 77 71 be 20 9a-0a 05 00 da 0c d5 eb c5 c$4wq. ......... 0030 - ce fa 8e 1c 4f e3 57 d7-e2 9f c1 51 a5 03 bc f2 ....O.W....Q.... 0040 - a3 03 dc 21 b9 f1 ba 75-2a 59 1a 3c 2f 74 8f 71 ...!...u*Y.

2.

Click here

``` sudo iotedge check --verbose Configuration checks (aziot-identity-service) --------------------------------------------- √ keyd configuration is well-formed - OK √ certd configuration is well-formed - OK √ tpmd configuration is well-formed - OK √ identityd configuration is well-formed - OK √ daemon configurations up-to-date with config.toml - OK √ identityd config toml file specifies a valid hostname - OK √ host time is close to reference time - OK √ preloaded certificates are valid - OK √ keyd is running - OK √ certd is running - OK √ identityd is running - OK √ read all preloaded certificates from the Certificates Service - OK √ read all preloaded key pairs from the Keys Service - OK √ ensure all preloaded certificates match preloaded private keys with the same ID - OK Connectivity checks (aziot-identity-service) -------------------------------------------- √ host can connect to and perform TLS handshake with iothub AMQP port - OK √ host can connect to and perform TLS handshake with iothub HTTPS / WebSockets port - OK √ host can connect to and perform TLS handshake with iothub MQTT port - OK Configuration checks -------------------- √ aziot-edged configuration is well-formed - OK √ configuration up-to-date with config.toml - OK √ container engine is installed and functional - OK √ configuration has correct parent_hostname - OK × configuration has correct URIs for daemon mgmt endpoint - Error SocketError - SocketErrorCode (AccessDenied) : Permission denied /var/run/iotedge/mgmt.sock One or more errors occurred. (Got bad response: ) caused by: docker returned exit code: 1, stderr = SocketError - SocketErrorCode (AccessDenied) : Permission denied /var/run/iotedge/mgmt.sock One or more errors occurred. (Got bad response: ) √ container time is close to host time - OK ‼ DNS server - Warning Container engine is not configured with DNS server setting, which may impact connectivity to IoT Hub. Please see https://aka.ms/iotedge-prod-checklist-dns for best practices. You can ignore this warning if you are setting DNS server per module in the Edge deployment. caused by: Could not open container engine config file /etc/docker/daemon.json caused by: No such file or directory (os error 2) ‼ production readiness: container engine - Warning Device is not using a production-supported container engine (moby-engine). Please see https://aka.ms/iotedge-prod-checklist-moby for details. ‼ production readiness: logs policy - Warning Container engine is not configured to rotate module logs which may cause it run out of disk space. Please see https://aka.ms/iotedge-prod-checklist-logs for best practices. You can ignore this warning if you are setting log policy per module in the Edge deployment. caused by: Could not open container engine config file /etc/docker/daemon.json caused by: No such file or directory (os error 2) × production readiness: Edge Agent's storage directory is persisted on the host filesystem - Error Could not check current state of edgeAgent container caused by: docker returned exit code: 1, stderr = Error: No such object: edgeAgent × production readiness: Edge Hub's storage directory is persisted on the host filesystem - Error Could not check current state of edgeHub container caused by: docker returned exit code: 1, stderr = Error: No such object: edgeHub √ Agent image is valid and can be pulled from upstream - OK Connectivity checks ------------------- √ container on the default network can connect to upstream AMQP port - OK × container on the default network can connect to upstream HTTPS / WebSockets port - Error Container on the default network could not connect to 192.168.1.90:443 caused by: docker returned exit code: 1, stderr = One or more errors occurred. (The given header was not found.) √ container on the default network can connect to upstream MQTT port - OK √ container on the IoT Edge module network can connect to upstream AMQP port - OK × container on the IoT Edge module network can connect to upstream HTTPS / WebSockets port - Error Container on the azure-iot-edge network could not connect to 192.168.1.90:443 caused by: docker returned exit code: 1, stderr = One or more errors occurred. (The given header was not found.) √ container on the IoT Edge module network can connect to upstream MQTT port - OK 27 check(s) succeeded. 3 check(s) raised warnings. 5 check(s) raised errors. ```

3.

Click here

``` [2021-08-24T09:35:11Z INFO api_proxy_module::monitors::config_monitor] Initializing config monitoring loop [2021-08-24T09:35:11Z INFO api_proxy_module::monitors::config_monitor] Starting config monitoring loop [2021-08-24T09:35:11Z INFO api_proxy_module::monitors::certs_monitor] Initializing certs monitoring loop [2021-08-24T09:35:12Z INFO api_proxy_module::monitors::certs_monitor] Starting certs monitoring loop [2021-08-24T09:35:13Z INFO api_proxy_module] Starting/Restarting API-Proxy [2021-08-24T09:35:59Z INFO api_proxy_module::token_service::token_client] Successfully generated new token 192.168.1.101 - - [24/Aug/2021:09:36:00 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 09:36:00 [info] 12#12: *1 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:09:36:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 09:36:59 [info] 12#12: *5 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:09:37:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 09:37:59 [info] 12#12: *7 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:09:38:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 09:38:59 [info] 12#12: *9 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:09:39:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 09:39:59 [info] 12#12: *11 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:09:40:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 09:40:59 [info] 12#12: *13 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:09:41:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 09:41:59 [info] 12#12: *15 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:09:42:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 09:42:59 [info] 12#12: *17 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:09:43:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 09:43:59 [info] 12#12: *19 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:09:44:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 09:44:59 [info] 12#12: *21 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:09:45:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 09:45:59 [info] 12#12: *23 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:09:46:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 09:46:59 [info] 12#12: *25 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:09:47:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 09:47:59 [info] 12#12: *27 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:09:48:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 09:48:59 [info] 12#12: *29 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:09:49:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 09:49:59 [info] 12#12: *31 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:09:50:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 09:50:59 [info] 12#12: *33 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:09:51:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 09:51:59 [info] 12#12: *35 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:09:52:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 09:52:59 [info] 12#12: *37 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:09:53:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 09:53:59 [info] 12#12: *39 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:09:54:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 09:54:59 [info] 12#12: *41 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:09:55:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 09:55:59 [info] 12#12: *43 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:09:56:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 09:56:59 [info] 12#12: *45 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:09:57:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 09:57:59 [info] 12#12: *47 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:09:58:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 09:58:59 [info] 12#12: *49 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:09:59:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 09:59:59 [info] 12#12: *51 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:00:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:00:59 [info] 12#12: *53 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:01:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:01:59 [info] 12#12: *55 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:02:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:02:59 [info] 12#12: *57 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:03:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:03:59 [info] 12#12: *59 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:04:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:04:59 [info] 12#12: *61 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:05:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:05:59 [info] 12#12: *63 client 192.168.1.101 closed keepalive connection [2021-08-24T10:06:59Z INFO api_proxy_module::token_service::token_client] Successfully generated new token 192.168.1.101 - - [24/Aug/2021:10:06:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:06:59 [info] 12#12: *65 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:07:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:07:59 [info] 12#12: *68 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:08:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:08:59 [info] 12#12: *70 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:09:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:09:59 [info] 12#12: *72 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:10:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:10:59 [info] 12#12: *74 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:11:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:11:59 [info] 12#12: *76 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:12:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:12:59 [info] 12#12: *78 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:13:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:13:59 [info] 12#12: *80 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:14:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:14:59 [info] 12#12: *82 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:15:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:15:59 [info] 12#12: *84 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:16:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:16:59 [info] 12#12: *86 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:17:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:17:59 [info] 12#12: *88 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:18:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:18:59 [info] 12#12: *90 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:19:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:19:59 [info] 12#12: *92 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:20:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:20:59 [info] 12#12: *94 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:21:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:21:59 [info] 12#12: *96 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:22:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:22:59 [info] 12#12: *98 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:23:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:23:59 [info] 12#12: *100 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:24:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:24:59 [info] 12#12: *102 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:25:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:25:59 [info] 12#12: *104 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:26:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:26:59 [info] 12#12: *106 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:27:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:27:59 [info] 12#12: *108 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:28:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:28:59 [info] 12#12: *110 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:29:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:29:59 [info] 12#12: *112 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:30:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:30:59 [info] 12#12: *114 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:31:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:31:59 [info] 12#12: *116 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:32:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:32:59 [info] 12#12: *118 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:33:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:33:59 [info] 12#12: *120 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:34:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:34:59 [info] 12#12: *122 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:35:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:35:59 [info] 12#12: *124 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:36:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:36:59 [info] 12#12: *126 client 192.168.1.101 closed keepalive connection [2021-08-24T10:37:59Z INFO api_proxy_module::token_service::token_client] Successfully generated new token 192.168.1.101 - - [24/Aug/2021:10:37:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:37:59 [info] 12#12: *128 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:38:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:38:59 [info] 12#12: *131 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:39:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:39:59 [info] 12#12: *133 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:40:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:40:59 [info] 12#12: *135 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:41:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:41:59 [info] 12#12: *137 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:42:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:42:59 [info] 12#12: *139 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:43:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:43:59 [info] 12#12: *141 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:44:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:44:59 [info] 12#12: *143 client 192.168.1.101 closed keepalive connection [2021-08-24T10:45:00Z WARN mqtt3::client] client will reconnect because of error: connection closed by server 192.168.1.101 - - [24/Aug/2021:10:45:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:45:59 [info] 12#12: *145 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:46:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:46:59 [info] 12#12: *147 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:47:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:47:59 [info] 12#12: *149 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:48:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:48:59 [info] 12#12: *151 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:49:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:49:59 [info] 12#12: *153 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:50:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:50:59 [info] 12#12: *155 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:51:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:51:59 [info] 12#12: *157 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:52:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:52:59 [info] 12#12: *159 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:53:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:53:59 [info] 12#12: *161 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:54:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:54:59 [info] 12#12: *163 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:55:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:55:59 [info] 12#12: *165 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:56:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:56:59 [info] 12#12: *167 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:57:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:57:59 [info] 12#12: *169 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:58:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:58:59 [info] 12#12: *171 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:10:59:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 10:59:59 [info] 12#12: *173 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:00:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:00:59 [info] 12#12: *175 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:01:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:01:59 [info] 12#12: *177 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:02:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:02:59 [info] 12#12: *179 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:03:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:03:59 [info] 12#12: *181 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:04:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:04:59 [info] 12#12: *183 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:05:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:05:59 [info] 12#12: *185 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:06:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:06:59 [info] 12#12: *187 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:07:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:07:59 [info] 12#12: *189 client 192.168.1.101 closed keepalive connection [2021-08-24T11:08:59Z INFO api_proxy_module::token_service::token_client] Successfully generated new token 192.168.1.101 - - [24/Aug/2021:11:08:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:08:59 [info] 12#12: *191 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:09:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:09:59 [info] 12#12: *194 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:10:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:10:59 [info] 12#12: *196 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:11:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:11:59 [info] 12#12: *198 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:12:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:12:59 [info] 12#12: *200 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:13:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:13:59 [info] 12#12: *202 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:14:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:14:59 [info] 12#12: *204 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:15:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:15:59 [info] 12#12: *206 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:16:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:16:59 [info] 12#12: *208 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:17:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:17:59 [info] 12#12: *210 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:18:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:18:59 [info] 12#12: *212 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:19:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:19:59 [info] 12#12: *214 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:20:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:20:59 [info] 12#12: *216 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:21:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:21:59 [info] 12#12: *218 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:22:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:22:59 [info] 12#12: *220 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:23:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:23:59 [info] 12#12: *222 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:24:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:24:59 [info] 12#12: *224 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:25:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:25:59 [info] 12#12: *226 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:26:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:26:59 [info] 12#12: *228 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:27:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:27:59 [info] 12#12: *230 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:28:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:28:59 [info] 12#12: *232 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:29:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:29:59 [info] 12#12: *234 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:30:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:30:59 [info] 12#12: *236 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:31:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:31:59 [info] 12#12: *238 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:32:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:32:59 [info] 12#12: *240 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:33:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:33:59 [info] 12#12: *242 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:34:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:34:59 [info] 12#12: *244 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:35:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:35:59 [info] 12#12: *246 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:36:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:36:59 [info] 12#12: *248 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:37:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:37:59 [info] 12#12: *250 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:38:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:38:59 [info] 12#12: *252 client 192.168.1.101 closed keepalive connection [2021-08-24T11:39:59Z INFO api_proxy_module::token_service::token_client] Successfully generated new token 192.168.1.101 - - [24/Aug/2021:11:39:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:39:59 [info] 12#12: *254 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:40:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:40:59 [info] 12#12: *257 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:41:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:41:59 [info] 12#12: *259 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:42:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:42:59 [info] 12#12: *261 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:43:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:43:59 [info] 12#12: *263 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:44:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:44:59 [info] 12#12: *265 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:45:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:45:59 [info] 12#12: *267 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:46:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:46:59 [info] 12#12: *269 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:47:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:47:59 [info] 12#12: *271 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:48:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:48:59 [info] 12#12: *273 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:49:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:49:59 [info] 12#12: *275 client 192.168.1.101 closed keepalive connection [2021-08-24T11:50:00Z WARN mqtt3::client] client will reconnect because of error: connection closed by server 192.168.1.101 - - [24/Aug/2021:11:50:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:50:59 [info] 12#12: *277 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:51:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:51:59 [info] 12#12: *279 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:52:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:52:59 [info] 12#12: *281 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:53:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:53:59 [info] 12#12: *283 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:54:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:54:59 [info] 12#12: *285 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:55:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:55:59 [info] 12#12: *287 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:56:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:56:59 [info] 12#12: *289 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:57:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:57:59 [info] 12#12: *291 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:58:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:58:59 [info] 12#12: *293 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:11:59:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 11:59:59 [info] 12#12: *295 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:00:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:00:59 [info] 12#12: *297 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:01:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:01:59 [info] 12#12: *299 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:02:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:02:59 [info] 12#12: *301 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:03:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:03:59 [info] 12#12: *303 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:04:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:04:59 [info] 12#12: *305 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:05:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:05:59 [info] 12#12: *307 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:06:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:06:59 [info] 12#12: *309 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:07:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:07:59 [info] 12#12: *311 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:08:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:08:59 [info] 12#12: *313 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:09:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:09:59 [info] 12#12: *315 client 192.168.1.101 closed keepalive connection [2021-08-24T12:10:59Z INFO api_proxy_module::token_service::token_client] Successfully generated new token 192.168.1.101 - - [24/Aug/2021:12:10:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:10:59 [info] 12#12: *317 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:11:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:11:59 [info] 12#12: *320 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:12:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:12:59 [info] 12#12: *322 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:13:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:13:59 [info] 12#12: *324 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:14:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:14:59 [info] 12#12: *326 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:15:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:15:59 [info] 12#12: *328 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:16:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:16:59 [info] 12#12: *330 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:17:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:17:59 [info] 12#12: *332 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:18:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:18:59 [info] 12#12: *334 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:19:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:19:59 [info] 12#12: *336 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:20:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:20:59 [info] 12#12: *338 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:21:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:21:59 [info] 12#12: *340 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:22:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:22:59 [info] 12#12: *342 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:23:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:23:59 [info] 12#12: *344 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:24:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:24:59 [info] 12#12: *346 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:25:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:25:59 [info] 12#12: *348 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:26:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:26:59 [info] 12#12: *350 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:27:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:27:59 [info] 12#12: *352 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:28:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:28:59 [info] 12#12: *354 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:29:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:29:59 [info] 12#12: *356 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:30:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:30:59 [info] 12#12: *358 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:31:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:31:59 [info] 12#12: *360 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:32:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:32:59 [info] 12#12: *362 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:33:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:33:59 [info] 12#12: *364 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:34:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:34:59 [info] 12#12: *366 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:35:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:35:59 [info] 12#12: *368 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:36:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:36:59 [info] 12#12: *370 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:37:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:37:59 [info] 12#12: *372 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:38:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:38:59 [info] 12#12: *374 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:39:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:39:59 [info] 12#12: *376 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:40:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:40:59 [info] 12#12: *378 client 192.168.1.101 closed keepalive connection [2021-08-24T12:41:59Z INFO api_proxy_module::token_service::token_client] Successfully generated new token 192.168.1.101 - - [24/Aug/2021:12:41:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:41:59 [info] 12#12: *380 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:42:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:42:59 [info] 12#12: *383 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:43:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:43:59 [info] 12#12: *385 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:44:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:44:59 [info] 12#12: *387 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:45:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:45:59 [info] 12#12: *389 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:46:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:46:59 [info] 12#12: *391 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:47:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:47:59 [info] 12#12: *393 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:48:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:48:59 [info] 12#12: *395 client 192.168.1.101 closed keepalive connection 2021/08/24 12:49:50 [info] 12#12: *397 client closed connection while waiting for request, client: 192.168.1.101, server: 0.0.0.0:443 192.168.1.101 - - [24/Aug/2021:12:49:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:49:59 [info] 12#12: *398 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:50:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:50:59 [info] 12#12: *400 client 192.168.1.101 closed keepalive connection 2021/08/24 12:51:35 [info] 12#12: *402 client closed connection while waiting for request, client: 192.168.1.101, server: 0.0.0.0:443 192.168.1.101 - - [24/Aug/2021:12:51:40 +0000] "GET /v2/ HTTP/1.1" 200 2 "-" "docker/20.10.5 go/go1.13.15 git-commit/363e9a8 kernel/5.4.24-iot-gate-imx8-2.4+gbabac008e5cf os/linux arch/arm64 UpstreamClient(Docker-Client/20.10.5 \x5C(linux\x5C))" 192.168.1.101 - - [24/Aug/2021:12:51:41 +0000] "HEAD /v2/azureiotedge-agent/manifests/1.2 HTTP/1.1" 200 0 "-" "docker/20.10.5 go/go1.13.15 git-commit/363e9a8 kernel/5.4.24-iot-gate-imx8-2.4+gbabac008e5cf os/linux arch/arm64 UpstreamClient(Docker-Client/20.10.5 \x5C(linux\x5C))" 192.168.1.101 - - [24/Aug/2021:12:51:46 +0000] "GET /devices/0000/modules HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:51:46 [info] 12#12: *408 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:51:53 +0000] "GET /devices/0000/modules HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:51:53 [info] 12#12: *410 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:51:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:51:59 [info] 12#12: *412 client 192.168.1.101 closed keepalive connection 192.168.1.101 - - [24/Aug/2021:12:52:59 +0000] "GET /devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview HTTP/1.1" 401 51 "-" "-" 2021/08/24 12:52:59 [info] 12#12: *414 client 192.168.1.101 closed keepalive connection ```

4.

Click here

``` 2021-08-24 09:34:43 +00:00 Starting Edge Hub 2021-08-24 09:34:43.642 +00:00 Edge Hub Main() <6> 2021-08-24 09:34:48.492 +00:00 [INF] - Installing certificates [CN=layer3.ca:06/08/2023 12:49:04],[CN=Azure_IoT_Hub_Intermediate_Cert_Test_Only:05/29/2023 11:56:52],[CN=Azure_IoT_Hub_CA_Cert_Test_Only:05/29/2023 11:56:50] to Root <6> 2021-08-24 09:34:48.685 +00:00 [INF] - Installing certificates [CN=layer3.ca:06/08/2023 12:49:04],[CN=Azure_IoT_Hub_Intermediate_Cert_Test_Only:05/29/2023 11:56:52],[CN=Azure_IoT_Hub_CA_Cert_Test_Only:05/29/2023 11:56:50],[CN=Azure_IoT_Hub_CA_Cert_Test_Only:05/29/2023 11:56:50] to Root <6> 2021-08-24 09:34:48.740 +00:00 [INF] - Enabling SSL protocols: Tls, Tls11, Tls12 <6> 2021-08-24 09:34:49.507 +00:00 [INF] - Experimental features configuration: {"Enabled":false,"DisableCloudSubscriptions":false,"DisableConnectivityCheck":false} <6> 2021-08-24 09:34:51.948 +00:00 [INF] - Created persistent store at /tmp/edgeHub <6> 2021-08-24 09:34:52.281 +00:00 [INF] - Initializing Edge Hub <6> 2021-08-24 09:34:52.283 +00:00 [INF] - █████╗ ███████╗██╗ ██╗██████╗ ███████╗ ██╔══██╗╚══███╔╝██║ ██║██╔══██╗██╔════╝ ███████║ ███╔╝ ██║ ██║██████╔╝█████╗ ██╔══██║ ███╔╝ ██║ ██║██╔══██╗██╔══╝ ██║ ██║███████╗╚██████╔╝██║ ██║███████╗ ╚═╝ ╚═╝╚══════╝ ╚═════╝ ╚═╝ ╚═╝╚══════╝ ██╗ ██████╗ ████████╗ ███████╗██████╗ ██████╗ ███████╗ ██║██╔═══██╗╚══██╔══╝ ██╔════╝██╔══██╗██╔════╝ ██╔════╝ ██║██║ ██║ ██║ █████╗ ██║ ██║██║ ███╗█████╗ ██║██║ ██║ ██║ ██╔══╝ ██║ ██║██║ ██║██╔══╝ ██║╚██████╔╝ ██║ ███████╗██████╔╝╚██████╔╝███████╗ ╚═╝ ╚═════╝ ╚═╝ ╚══════╝╚═════╝ ╚═════╝ ╚══════╝ <6> 2021-08-24 09:34:52.290 +00:00 [INF] - Version - 1.1.4.44136317 (cbeb53cf8b5889718c4724936f6d13ef5ceaac2e) <6> 2021-08-24 09:34:52.291 +00:00 [INF] - OptimizeForPerformance=False <6> 2021-08-24 09:34:52.290 +00:00 [INF] - Created device scope identities cache <6> 2021-08-24 09:34:52.291 +00:00 [INF] - MessageAckTimeoutSecs=30 <6> 2021-08-24 09:34:52.295 +00:00 [INF] - Loaded server certificate with expiration date of "2021-09-23T09:34:46.0000000+00:00" <6> 2021-08-24 09:34:52.324 +00:00 [INF] - Starting refresh of device scope identities cache <6> 2021-08-24 09:34:52.336 +00:00 [INF] - Using Asp Net server for metrics <6> 2021-08-24 09:34:52.617 +00:00 [INF] - Started task to cleanup processed and stale messages <6> 2021-08-24 09:34:52.617 +00:00 [INF] - Created new message store <6> 2021-08-24 09:34:52.980 +00:00 [INF] - Created DeviceConnectivityManager with connected check frequency 00:05:00 and disconnected check frequency 00:02:00 <6> 2021-08-24 09:34:53.124 +00:00 [INF] - Initialized storing twin manager <6> 2021-08-24 09:34:53.191 +00:00 [INF] - Initializing configuration <6> 2021-08-24 09:34:53.266 +00:00 [INF] - New device connection for device red-iot-layer3-edge/$edgeHub <6> 2021-08-24 09:34:53.339 +00:00 [INF] - Client red-iot-layer3-edge/$edgeHub connected to edgeHub, processing existing subscriptions. <6> 2021-08-24 09:34:53.669 +00:00 [INF] - Processing pending subscriptions for red-iot-layer3-edge/$edgeHub <6> 2021-08-24 09:34:54.912 +00:00 [INF] - Attempting to connect to IoT Hub for client red-iot-layer3-edge/$edgeHub via AMQP... <4> 2021-08-24 09:34:55.169 +00:00 [WRN] - Empty edge hub configuration received. Ignoring... <6> 2021-08-24 09:34:57.437 +00:00 [INF] - Exiting disconnected state <6> 2021-08-24 09:34:57.514 +00:00 [INF] - Received device connected callback <6> 2021-08-24 09:34:57.557 +00:00 [INF] - Device connected to cloud, processing subscriptions for connected clients. <6> 2021-08-24 09:34:57.568 +00:00 [INF] - Processing subscriptions for client red-iot-layer3-edge/$edgeHub on device connected to cloud. <6> 2021-08-24 09:34:57.570 +00:00 [INF] - Skipping red-iot-layer3-edge/$edgeHub for subscription processing, as it is currently being processed. <6> 2021-08-24 09:34:57.622 +00:00 [INF] - Entering connected state <6> 2021-08-24 09:34:57.638 +00:00 [INF] - Cloud connection for red-iot-layer3-edge/$edgeHub is True <6> 2021-08-24 09:34:57.652 +00:00 [INF] - Connection status for red-iot-layer3-edge/$edgeHub changed to ConnectionEstablished <6> 2021-08-24 09:34:57.659 +00:00 [INF] - Client red-iot-layer3-edge/$edgeHub connected to cloud, processing existing subscriptions. <6> 2021-08-24 09:34:57.660 +00:00 [INF] - Skipping red-iot-layer3-edge/$edgeHub for subscription processing, as it is currently being processed. <6> 2021-08-24 09:34:57.675 +00:00 [INF] - Created cloud proxy for client red-iot-layer3-edge/$edgeHub via AMQP, with client operation timeout 20 seconds. <6> 2021-08-24 09:34:57.702 +00:00 [INF] - Initialized cloud proxy b008fa3f-3456-4d75-b5ac-2618f4a31488 for red-iot-layer3-edge/$edgeHub <6> 2021-08-24 09:34:57.715 +00:00 [INF] - Created cloud connection for client red-iot-layer3-edge/$edgeHub <6> 2021-08-24 09:34:58.500 +00:00 [INF] - Processing pending subscriptions for red-iot-layer3-edge/$edgeHub <6> 2021-08-24 09:34:58.510 +00:00 [INF] - Updated reported properties for red-iot-layer3-edge/$edgeHub <6> 2021-08-24 09:34:59.559 +00:00 [INF] - Created cloud endpoint iothub with max batch size 10 and fan-out factor of 10. <6> 2021-08-24 09:35:00.015 +00:00 [INF] - Obtained edge hub config from module twin <6> 2021-08-24 09:35:00.711 +00:00 [INF] - Set the following 1 route(s) in edge hub <6> 2021-08-24 09:35:00.711 +00:00 [INF] - route: FROM /messages/* INTO $upstream <6> 2021-08-24 09:35:00.717 +00:00 [INF] - Updated message store TTL to 7200 seconds <6> 2021-08-24 09:35:00.719 +00:00 [INF] - Updated the edge hub store and forward configuration <6> 2021-08-24 09:35:00.749 +00:00 [INF] - Started operation Get EdgeHub config <6> 2021-08-24 09:35:00.751 +00:00 [INF] - Initialized edge hub configuration <6> 2021-08-24 09:35:00.786 +00:00 [INF] - Starting timer to authenticate connections with a period of 300 seconds <6> 2021-08-24 09:35:01.098 +00:00 [INF] - Scheduling server certificate renewal for "2021-09-23T09:32:16.0008939Z". <6> 2021-08-24 09:35:01.104 +00:00 [INF] - Starting protocol heads - (MQTT, AMQP, HTTP) <6> 2021-08-24 09:35:01.124 +00:00 [INF] - Starting MQTT head <6> 2021-08-24 09:35:01.264 +00:00 [INF] - Initializing TLS endpoint on port 8883 for MQTT head. <6> 2021-08-24 09:35:01.570 +00:00 [INF] - Starting AMQP head <6> 2021-08-24 09:35:01.609 +00:00 [INF] - Started MQTT head <6> 2021-08-24 09:35:01.810 +00:00 [INF] - Started AMQP head <6> 2021-08-24 09:35:01.816 +00:00 [INF] - Starting HTTP head <6> 2021-08-24 09:35:04.349 +00:00 [INF] - Updated reported properties for red-iot-layer3-edge/$edgeHub <4> 2021-08-24 09:35:07.225 +00:00 [WRN] - Overriding address(es) '"http://+:80"'. Binding to endpoints defined in "UseKestrel()" instead. <6> 2021-08-24 09:35:07.249 +00:00 [INF] - Started HTTP head <6> 2021-08-24 09:35:11.714 +00:00 [INF] - Client red-iot-layer3-edge/IoTEdgeAPIProxy in device scope authenticated locally. <6> 2021-08-24 09:35:11.762 +00:00 [INF] - Successfully generated identity for clientId red-iot-layer3-edge/IoTEdgeAPIProxy and username 192.168.1.90/red-iot-layer3-edge/IoTEdgeAPIProxy/?api-version=2018-06-30 <6> 2021-08-24 09:35:11.783 +00:00 [INF] - ClientAuthenticated, red-iot-layer3-edge/IoTEdgeAPIProxy, 76ad33bd <6> 2021-08-24 09:35:11.891 +00:00 [INF] - New device connection for device red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 09:35:11.892 +00:00 [INF] - Client red-iot-layer3-edge/IoTEdgeAPIProxy connected to edgeHub, processing existing subscriptions. <6> 2021-08-24 09:35:11.960 +00:00 [INF] - Bind device proxy for device red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 09:35:11.963 +00:00 [INF] - Binding message channel for device Id red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 09:35:12.076 +00:00 [INF] - Processing subscriptions Methods, DesiredPropertyUpdates, TwinResponse for client red-iot-layer3-edge/IoTEdgeAPIProxy. <6> 2021-08-24 09:35:12.077 +00:00 [INF] - Processing pending subscriptions for red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 09:35:12.081 +00:00 [INF] - Set subscriptions from session state for red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 09:35:12.110 +00:00 [INF] - Attempting to connect to IoT Hub for client red-iot-layer3-edge/IoTEdgeAPIProxy via AMQP... <6> 2021-08-24 09:35:12.613 +00:00 [INF] - Cloud connection for red-iot-layer3-edge/IoTEdgeAPIProxy is True <6> 2021-08-24 09:35:12.614 +00:00 [INF] - Connection status for red-iot-layer3-edge/IoTEdgeAPIProxy changed to ConnectionEstablished <6> 2021-08-24 09:35:12.614 +00:00 [INF] - Client red-iot-layer3-edge/IoTEdgeAPIProxy connected to cloud, processing existing subscriptions. <6> 2021-08-24 09:35:12.614 +00:00 [INF] - Skipping red-iot-layer3-edge/IoTEdgeAPIProxy for subscription processing, as it is currently being processed. <6> 2021-08-24 09:35:12.615 +00:00 [INF] - Created cloud proxy for client red-iot-layer3-edge/IoTEdgeAPIProxy via AMQP, with client operation timeout 20 seconds. <6> 2021-08-24 09:35:12.616 +00:00 [INF] - Initialized cloud proxy 6367bb4d-1b24-48d3-b814-9027a0bf150a for red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 09:35:12.616 +00:00 [INF] - Created cloud connection for client red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 09:35:14.929 +00:00 [INF] - Updated reported properties for red-iot-layer3-edge/$edgeHub <6> 2021-08-24 09:35:15.230 +00:00 [INF] - Done syncing reported properties for red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 09:35:19.141 +00:00 [INF] - Client red-iot-layer3-edge/modbus in device scope authenticated locally. <6> 2021-08-24 09:35:19.145 +00:00 [INF] - Successfully generated identity for clientId red-iot-layer3-edge/modbus and username 192.168.1.90/red-iot-layer3-edge/modbus/?api-version=2020-09-30&DeviceClientType=.NET%2F1.37.2%20%28.NET%20Core%203.1.17%3B%20Linux%205.4.24-iot-gate-imx8-2.4%2Bgbabac008e5cf%20%231%20SMP%20PREEMPT%20Tue%20Mar%202%2012%3A34%3A06%20UTC%202021%3B%20Arm64%29 <6> 2021-08-24 09:35:19.146 +00:00 [INF] - ClientAuthenticated, red-iot-layer3-edge/modbus, 35857cee <6> 2021-08-24 09:35:19.153 +00:00 [INF] - New device connection for device red-iot-layer3-edge/modbus <6> 2021-08-24 09:35:19.156 +00:00 [INF] - Client red-iot-layer3-edge/modbus connected to edgeHub, processing existing subscriptions. <6> 2021-08-24 09:35:19.169 +00:00 [INF] - Bind device proxy for device red-iot-layer3-edge/modbus <6> 2021-08-24 09:35:19.170 +00:00 [INF] - Binding message channel for device Id red-iot-layer3-edge/modbus <6> 2021-08-24 09:35:19.172 +00:00 [INF] - Attempting to connect to IoT Hub for client red-iot-layer3-edge/modbus via AMQP... <6> 2021-08-24 09:35:19.331 +00:00 [INF] - Processing subscriptions TwinResponse for client red-iot-layer3-edge/modbus. <6> 2021-08-24 09:35:19.334 +00:00 [INF] - Processing pending subscriptions for red-iot-layer3-edge/modbus <6> 2021-08-24 09:35:19.451 +00:00 [INF] - Set subscriptions from session state for red-iot-layer3-edge/modbus <6> 2021-08-24 09:35:19.541 +00:00 [INF] - Cloud connection for red-iot-layer3-edge/modbus is True <6> 2021-08-24 09:35:19.541 +00:00 [INF] - Connection status for red-iot-layer3-edge/modbus changed to ConnectionEstablished <6> 2021-08-24 09:35:19.542 +00:00 [INF] - Client red-iot-layer3-edge/modbus connected to cloud, processing existing subscriptions. <6> 2021-08-24 09:35:19.542 +00:00 [INF] - Skipping red-iot-layer3-edge/modbus for subscription processing, as it is currently being processed. <6> 2021-08-24 09:35:19.542 +00:00 [INF] - Created cloud proxy for client red-iot-layer3-edge/modbus via AMQP, with client operation timeout 20 seconds. <6> 2021-08-24 09:35:19.543 +00:00 [INF] - Initialized cloud proxy 1ece64fd-a8dd-4f2b-bb0a-bdc7aedc7640 for red-iot-layer3-edge/modbus <6> 2021-08-24 09:35:19.543 +00:00 [INF] - Created cloud connection for client red-iot-layer3-edge/modbus <6> 2021-08-24 09:35:19.737 +00:00 [INF] - Processing subscriptions TwinResponse, ModuleMessages for client red-iot-layer3-edge/modbus. <6> 2021-08-24 09:35:19.740 +00:00 [INF] - Processing pending subscriptions for red-iot-layer3-edge/modbus <6> 2021-08-24 09:35:19.742 +00:00 [INF] - Set subscriptions from session state for red-iot-layer3-edge/modbus <6> 2021-08-24 09:35:20.557 +00:00 [INF] - Updated reported properties for red-iot-layer3-edge/$edgeHub <6> 2021-08-24 09:35:59.947 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 09:36:00.029 +00:00 [INF] - Request finished in 84.6835ms 401 text/html <6> 2021-08-24 09:36:59.465 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 09:36:59.470 +00:00 [INF] - Request finished in 6.0221ms 401 text/html <6> 2021-08-24 09:37:59.454 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 09:37:59.458 +00:00 [INF] - Request finished in 4.948ms 401 text/html <6> 2021-08-24 09:38:59.452 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 09:38:59.456 +00:00 [INF] - Request finished in 4.1694ms 401 text/html <6> 2021-08-24 09:39:59.624 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 09:39:59.627 +00:00 [INF] - Request finished in 3.1992ms 401 text/html <6> 2021-08-24 09:40:00.800 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 09:40:59.452 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 09:40:59.456 +00:00 [INF] - Request finished in 3.9297ms 401 text/html <6> 2021-08-24 09:41:59.451 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 09:41:59.454 +00:00 [INF] - Request finished in 3.3358ms 401 text/html <6> 2021-08-24 09:42:59.449 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 09:42:59.452 +00:00 [INF] - Request finished in 3.0533ms 401 text/html <6> 2021-08-24 09:43:59.449 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 09:43:59.452 +00:00 [INF] - Request finished in 2.9625ms 401 text/html <6> 2021-08-24 09:44:59.462 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 09:44:59.465 +00:00 [INF] - Request finished in 2.6288ms 401 text/html <6> 2021-08-24 09:45:00.785 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 09:45:59.656 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 09:45:59.658 +00:00 [INF] - Request finished in 2.3305ms 401 text/html <6> 2021-08-24 09:46:59.449 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 09:46:59.451 +00:00 [INF] - Request finished in 2.3937ms 401 text/html <6> 2021-08-24 09:47:59.447 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 09:47:59.450 +00:00 [INF] - Request finished in 2.1308ms 401 text/html <6> 2021-08-24 09:48:59.446 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 09:48:59.449 +00:00 [INF] - Request finished in 2.4349ms 401 text/html <6> 2021-08-24 09:49:59.453 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 09:49:59.455 +00:00 [INF] - Request finished in 2.0395ms 401 text/html <6> 2021-08-24 09:50:00.784 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 09:50:59.448 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 09:50:59.450 +00:00 [INF] - Request finished in 2.3316ms 401 text/html <6> 2021-08-24 09:51:59.600 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 09:51:59.602 +00:00 [INF] - Request finished in 2.1627ms 401 text/html <6> 2021-08-24 09:52:59.451 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 09:52:59.454 +00:00 [INF] - Request finished in 2.229ms 401 text/html <6> 2021-08-24 09:53:59.444 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 09:53:59.446 +00:00 [INF] - Request finished in 2.3672ms 401 text/html <6> 2021-08-24 09:54:59.445 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 09:54:59.447 +00:00 [INF] - Request finished in 1.8872ms 401 text/html <6> 2021-08-24 09:55:00.783 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 09:55:59.441 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 09:55:59.443 +00:00 [INF] - Request finished in 1.6097ms 401 text/html <6> 2021-08-24 09:56:59.444 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 09:56:59.446 +00:00 [INF] - Request finished in 1.7009ms 401 text/html <6> 2021-08-24 09:57:59.631 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 09:57:59.633 +00:00 [INF] - Request finished in 2.0254ms 401 text/html <6> 2021-08-24 09:58:59.447 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 09:58:59.448 +00:00 [INF] - Request finished in 1.6221ms 401 text/html <6> 2021-08-24 09:59:59.445 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 09:59:59.447 +00:00 [INF] - Request finished in 1.8354ms 401 text/html <6> 2021-08-24 10:00:00.786 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 10:00:59.445 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:00:59.447 +00:00 [INF] - Request finished in 1.9842ms 401 text/html <6> 2021-08-24 10:01:59.447 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:01:59.449 +00:00 [INF] - Request finished in 1.9338ms 401 text/html <6> 2021-08-24 10:02:59.445 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:02:59.447 +00:00 [INF] - Request finished in 1.7445ms 401 text/html <6> 2021-08-24 10:03:59.607 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:03:59.609 +00:00 [INF] - Request finished in 2.1505ms 401 text/html <6> 2021-08-24 10:04:52.693 +00:00 [INF] - Started task to cleanup processed and stale messages for endpoint iothub <6> 2021-08-24 10:04:52.709 +00:00 [INF] - Cleaned up 0 messages from queue for endpoint iothub and 0 messages from message store. <6> 2021-08-24 10:04:59.450 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:04:59.452 +00:00 [INF] - Request finished in 2.329ms 401 text/html <6> 2021-08-24 10:05:00.784 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 10:05:59.442 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:05:59.444 +00:00 [INF] - Request finished in 1.2394ms 401 text/html <6> 2021-08-24 10:06:59.690 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:06:59.691 +00:00 [INF] - Request finished in 0.6628ms 401 text/html <6> 2021-08-24 10:07:59.450 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:07:59.450 +00:00 [INF] - Request finished in 0.6395ms 401 text/html <6> 2021-08-24 10:08:59.441 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:08:59.442 +00:00 [INF] - Request finished in 0.6454ms 401 text/html <6> 2021-08-24 10:09:59.602 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:09:59.603 +00:00 [INF] - Request finished in 0.6657ms 401 text/html <6> 2021-08-24 10:10:00.786 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 10:10:59.440 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:10:59.440 +00:00 [INF] - Request finished in 0.6836ms 401 text/html <6> 2021-08-24 10:11:59.441 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:11:59.441 +00:00 [INF] - Request finished in 0.629ms 401 text/html <6> 2021-08-24 10:12:59.441 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:12:59.441 +00:00 [INF] - Request finished in 0.6321ms 401 text/html <6> 2021-08-24 10:13:59.464 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:13:59.464 +00:00 [INF] - Request finished in 0.6555ms 401 text/html <6> 2021-08-24 10:14:59.442 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:14:59.442 +00:00 [INF] - Request finished in 0.6709ms 401 text/html <6> 2021-08-24 10:15:00.786 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 10:15:59.607 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:15:59.608 +00:00 [INF] - Request finished in 0.7125ms 401 text/html <6> 2021-08-24 10:16:59.441 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:16:59.442 +00:00 [INF] - Request finished in 0.6326ms 401 text/html <6> 2021-08-24 10:17:59.441 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:17:59.442 +00:00 [INF] - Request finished in 0.6441ms 401 text/html <6> 2021-08-24 10:18:59.441 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:18:59.442 +00:00 [INF] - Request finished in 0.6781ms 401 text/html <6> 2021-08-24 10:19:59.441 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:19:59.442 +00:00 [INF] - Request finished in 0.6757ms 401 text/html <6> 2021-08-24 10:20:00.788 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 10:20:59.442 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:20:59.442 +00:00 [INF] - Request finished in 0.6696ms 401 text/html <6> 2021-08-24 10:21:59.595 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:21:59.595 +00:00 [INF] - Request finished in 0.6655ms 401 text/html <6> 2021-08-24 10:22:59.441 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:22:59.442 +00:00 [INF] - Request finished in 0.6499ms 401 text/html <6> 2021-08-24 10:23:59.441 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:23:59.442 +00:00 [INF] - Request finished in 0.6446ms 401 text/html <6> 2021-08-24 10:24:59.441 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:24:59.442 +00:00 [INF] - Request finished in 0.6321ms 401 text/html <6> 2021-08-24 10:25:00.788 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 10:25:59.440 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:25:59.441 +00:00 [INF] - Request finished in 0.6359ms 401 text/html <6> 2021-08-24 10:26:59.441 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:26:59.441 +00:00 [INF] - Request finished in 0.6597ms 401 text/html <6> 2021-08-24 10:27:59.594 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:27:59.595 +00:00 [INF] - Request finished in 0.7089ms 401 text/html <6> 2021-08-24 10:28:59.440 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:28:59.441 +00:00 [INF] - Request finished in 0.7675ms 401 text/html <6> 2021-08-24 10:29:59.438 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:29:59.439 +00:00 [INF] - Request finished in 0.6706ms 401 text/html <6> 2021-08-24 10:30:00.784 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 10:30:59.441 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:30:59.442 +00:00 [INF] - Request finished in 0.6881ms 401 text/html <6> 2021-08-24 10:31:59.442 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:31:59.442 +00:00 [INF] - Request finished in 0.6422ms 401 text/html <6> 2021-08-24 10:32:59.440 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:32:59.441 +00:00 [INF] - Request finished in 0.6385ms 401 text/html <6> 2021-08-24 10:33:59.603 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:33:59.603 +00:00 [INF] - Request finished in 0.7866ms 401 text/html <6> 2021-08-24 10:34:36.141 +00:00 [INF] - Request starting HTTP/1.1 GET http://172.18.0.3:9600/metrics <6> 2021-08-24 10:34:36.261 +00:00 [INF] - Request finished in 120.0523ms 200 text/plain; version=0.0.4; charset=utf-8 <6> 2021-08-24 10:34:55.457 +00:00 [INF] - Starting refresh of device scope identities cache <6> 2021-08-24 10:34:59.433 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:34:59.439 +00:00 [INF] - Request finished in 5.7957ms 401 text/html <6> 2021-08-24 10:35:00.746 +00:00 [INF] - Starting periodic operation Get EdgeHub config... <6> 2021-08-24 10:35:00.785 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 10:35:01.217 +00:00 [INF] - Obtained edge hub config from module twin <6> 2021-08-24 10:35:01.411 +00:00 [INF] - Set the following 1 route(s) in edge hub <6> 2021-08-24 10:35:01.412 +00:00 [INF] - route: FROM /messages/* INTO $upstream <6> 2021-08-24 10:35:01.412 +00:00 [INF] - Updated message store TTL to 7200 seconds <6> 2021-08-24 10:35:01.412 +00:00 [INF] - Updated the edge hub store and forward configuration <6> 2021-08-24 10:35:01.412 +00:00 [INF] - Successfully completed periodic operation Get EdgeHub config <6> 2021-08-24 10:35:01.509 +00:00 [INF] - Updated reported properties for red-iot-layer3-edge/$edgeHub <6> 2021-08-24 10:35:19.544 +00:00 [INF] - Closing cloud proxy 1ece64fd-a8dd-4f2b-bb0a-bdc7aedc7640 for red-iot-layer3-edge/modbus because of inactivity <6> 2021-08-24 10:35:19.603 +00:00 [INF] - Closing receiver in cloud proxy 1ece64fd-a8dd-4f2b-bb0a-bdc7aedc7640 for red-iot-layer3-edge/modbus <6> 2021-08-24 10:35:19.604 +00:00 [INF] - Closed cloud proxy 1ece64fd-a8dd-4f2b-bb0a-bdc7aedc7640 for red-iot-layer3-edge/modbus <6> 2021-08-24 10:35:22.711 +00:00 [INF] - Started task to cleanup processed and stale messages for endpoint iothub <6> 2021-08-24 10:35:22.712 +00:00 [INF] - Cleaned up 0 messages from queue for endpoint iothub and 0 messages from message store. <6> 2021-08-24 10:35:59.441 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:35:59.441 +00:00 [INF] - Request finished in 0.6595ms 401 text/html <6> 2021-08-24 10:36:59.441 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:36:59.441 +00:00 [INF] - Request finished in 0.6415ms 401 text/html <6> 2021-08-24 10:37:59.684 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:37:59.684 +00:00 [INF] - Request finished in 0.6155ms 401 text/html <6> 2021-08-24 10:38:59.444 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:38:59.445 +00:00 [INF] - Request finished in 0.6426ms 401 text/html <6> 2021-08-24 10:39:59.596 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:39:59.597 +00:00 [INF] - Request finished in 0.688ms 401 text/html <6> 2021-08-24 10:40:00.785 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 10:40:59.440 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:40:59.441 +00:00 [INF] - Request finished in 0.6584ms 401 text/html <6> 2021-08-24 10:41:59.441 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:41:59.441 +00:00 [INF] - Request finished in 0.6407ms 401 text/html <6> 2021-08-24 10:42:59.441 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:42:59.441 +00:00 [INF] - Request finished in 0.6504ms 401 text/html <6> 2021-08-24 10:43:59.436 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:43:59.436 +00:00 [INF] - Request finished in 0.643ms 401 text/html <6> 2021-08-24 10:44:59.441 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:44:59.442 +00:00 [INF] - Request finished in 0.6408ms 401 text/html <6> 2021-08-24 10:45:00.786 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 10:45:00.791 +00:00 [INF] - Error authenticating token for red-iot-layer3-edge/IoTEdgeAPIProxy because the token is expired or could not be parsed <6> 2021-08-24 10:45:00.792 +00:00 [INF] - Credentials for client red-iot-layer3-edge/IoTEdgeAPIProxy are not valid. <4> 2021-08-24 10:45:00.793 +00:00 [WRN] - Reauthenticating client red-iot-layer3-edge/IoTEdgeAPIProxy failed, removing client connection <6> 2021-08-24 10:45:00.794 +00:00 [INF] - Unable to re-authenticate red-iot-layer3-edge/IoTEdgeAPIProxy, dropping client connection. <6> 2021-08-24 10:45:00.802 +00:00 [INF] - Removing device connection for device red-iot-layer3-edge/IoTEdgeAPIProxy with removeCloudConnection flag 'True'. <4> 2021-08-24 10:45:00.816 +00:00 [WRN] - Closing connection for device: red-iot-layer3-edge/IoTEdgeAPIProxy, Microsoft.Azure.Devices.Edge.Hub.Core.EdgeHubConnectionException: Connection closed for device red-iot-layer3-edge/IoTEdgeAPIProxy., <6> 2021-08-24 10:45:00.820 +00:00 [INF] - Disposing MessagingServiceClient for device Id red-iot-layer3-edge/IoTEdgeAPIProxy because of exception - Microsoft.Azure.Devices.Edge.Hub.Core.EdgeHubConnectionException: Connection closed for device red-iot-layer3-edge/IoTEdgeAPIProxy. <6> 2021-08-24 10:45:00.840 +00:00 [INF] - Closing device proxy for device Id red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 10:45:00.855 +00:00 [INF] - Closing receiver in cloud proxy 6367bb4d-1b24-48d3-b814-9027a0bf150a for red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 10:45:00.855 +00:00 [INF] - Closed cloud proxy 6367bb4d-1b24-48d3-b814-9027a0bf150a for red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 10:45:00.857 +00:00 [INF] - Device connection removed for device red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 10:45:00.874 +00:00 [INF] - Error authenticating token for red-iot-layer3-edge/modbus because the token is expired or could not be parsed <6> 2021-08-24 10:45:00.875 +00:00 [INF] - Credentials for client red-iot-layer3-edge/modbus are not valid. <4> 2021-08-24 10:45:00.875 +00:00 [WRN] - Reauthenticating client red-iot-layer3-edge/modbus failed, removing client connection <6> 2021-08-24 10:45:00.876 +00:00 [INF] - Unable to re-authenticate red-iot-layer3-edge/modbus, dropping client connection. <6> 2021-08-24 10:45:00.876 +00:00 [INF] - Removing device connection for device red-iot-layer3-edge/modbus with removeCloudConnection flag 'True'. <4> 2021-08-24 10:45:00.878 +00:00 [WRN] - Closing connection for device: red-iot-layer3-edge/modbus, Microsoft.Azure.Devices.Edge.Hub.Core.EdgeHubConnectionException: Connection closed for device red-iot-layer3-edge/modbus., <6> 2021-08-24 10:45:00.878 +00:00 [INF] - Disposing MessagingServiceClient for device Id red-iot-layer3-edge/modbus because of exception - Microsoft.Azure.Devices.Edge.Hub.Core.EdgeHubConnectionException: Connection closed for device red-iot-layer3-edge/modbus. <6> 2021-08-24 10:45:00.883 +00:00 [INF] - Closing device proxy for device Id red-iot-layer3-edge/modbus <6> 2021-08-24 10:45:00.883 +00:00 [INF] - Device connection removed for device red-iot-layer3-edge/modbus <6> 2021-08-24 10:45:01.635 +00:00 [INF] - Updated reported properties for red-iot-layer3-edge/$edgeHub <6> 2021-08-24 10:45:01.832 +00:00 [INF] - Client red-iot-layer3-edge/IoTEdgeAPIProxy in device scope authenticated locally. <6> 2021-08-24 10:45:01.834 +00:00 [INF] - Successfully generated identity for clientId red-iot-layer3-edge/IoTEdgeAPIProxy and username 192.168.1.90/red-iot-layer3-edge/IoTEdgeAPIProxy/?api-version=2018-06-30 <6> 2021-08-24 10:45:01.834 +00:00 [INF] - ClientAuthenticated, red-iot-layer3-edge/IoTEdgeAPIProxy, 687d03ab <6> 2021-08-24 10:45:01.838 +00:00 [INF] - New device connection for device red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 10:45:01.839 +00:00 [INF] - Client red-iot-layer3-edge/IoTEdgeAPIProxy connected to edgeHub, processing existing subscriptions. <6> 2021-08-24 10:45:01.844 +00:00 [INF] - Bind device proxy for device red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 10:45:01.844 +00:00 [INF] - Binding message channel for device Id red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 10:45:01.846 +00:00 [INF] - Processing subscriptions Methods, DesiredPropertyUpdates, TwinResponse for client red-iot-layer3-edge/IoTEdgeAPIProxy. <6> 2021-08-24 10:45:01.848 +00:00 [INF] - Attempting to connect to IoT Hub for client red-iot-layer3-edge/IoTEdgeAPIProxy via AMQP... <6> 2021-08-24 10:45:01.851 +00:00 [INF] - Processing pending subscriptions for red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 10:45:01.856 +00:00 [INF] - Set subscriptions from session state for red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 10:45:01.940 +00:00 [INF] - Client red-iot-layer3-edge/modbus in device scope authenticated locally. <6> 2021-08-24 10:45:01.942 +00:00 [INF] - Successfully generated identity for clientId red-iot-layer3-edge/modbus and username 192.168.1.90/red-iot-layer3-edge/modbus/?api-version=2020-09-30&DeviceClientType=.NET%2F1.37.2%20%28.NET%20Core%203.1.17%3B%20Linux%205.4.24-iot-gate-imx8-2.4%2Bgbabac008e5cf%20%231%20SMP%20PREEMPT%20Tue%20Mar%202%2012%3A34%3A06%20UTC%202021%3B%20Arm64%29 <6> 2021-08-24 10:45:01.943 +00:00 [INF] - ClientAuthenticated, red-iot-layer3-edge/modbus, 23be341a <6> 2021-08-24 10:45:01.973 +00:00 [INF] - New device connection for device red-iot-layer3-edge/modbus <6> 2021-08-24 10:45:01.973 +00:00 [INF] - Client red-iot-layer3-edge/modbus connected to edgeHub, processing existing subscriptions. <6> 2021-08-24 10:45:01.975 +00:00 [INF] - Bind device proxy for device red-iot-layer3-edge/modbus <6> 2021-08-24 10:45:01.975 +00:00 [INF] - Binding message channel for device Id red-iot-layer3-edge/modbus <6> 2021-08-24 10:45:01.984 +00:00 [INF] - Attempting to connect to IoT Hub for client red-iot-layer3-edge/modbus via AMQP... <6> 2021-08-24 10:45:01.995 +00:00 [INF] - Processing subscriptions TwinResponse, ModuleMessages for client red-iot-layer3-edge/modbus. <6> 2021-08-24 10:45:01.998 +00:00 [INF] - Set subscriptions from session state for red-iot-layer3-edge/modbus <6> 2021-08-24 10:45:02.010 +00:00 [INF] - Processing subscriptions TwinResponse, ModuleMessages for client red-iot-layer3-edge/modbus. <6> 2021-08-24 10:45:02.011 +00:00 [INF] - Set subscriptions from session state for red-iot-layer3-edge/modbus <6> 2021-08-24 10:45:02.199 +00:00 [INF] - Cloud connection for red-iot-layer3-edge/IoTEdgeAPIProxy is True <6> 2021-08-24 10:45:02.200 +00:00 [INF] - Connection status for red-iot-layer3-edge/IoTEdgeAPIProxy changed to ConnectionEstablished <6> 2021-08-24 10:45:02.200 +00:00 [INF] - Client red-iot-layer3-edge/IoTEdgeAPIProxy connected to cloud, processing existing subscriptions. <6> 2021-08-24 10:45:02.201 +00:00 [INF] - Skipping red-iot-layer3-edge/IoTEdgeAPIProxy for subscription processing, as it is currently being processed. <6> 2021-08-24 10:45:02.201 +00:00 [INF] - Created cloud proxy for client red-iot-layer3-edge/IoTEdgeAPIProxy via AMQP, with client operation timeout 20 seconds. <6> 2021-08-24 10:45:02.201 +00:00 [INF] - Initialized cloud proxy 18aa625d-f72c-4f8e-a35c-69d0f027fdaf for red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 10:45:02.202 +00:00 [INF] - Created cloud connection for client red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 10:45:02.275 +00:00 [INF] - Processing pending subscriptions for red-iot-layer3-edge/modbus <6> 2021-08-24 10:45:02.326 +00:00 [INF] - Cloud connection for red-iot-layer3-edge/modbus is True <6> 2021-08-24 10:45:02.327 +00:00 [INF] - Connection status for red-iot-layer3-edge/modbus changed to ConnectionEstablished <6> 2021-08-24 10:45:02.327 +00:00 [INF] - Client red-iot-layer3-edge/modbus connected to cloud, processing existing subscriptions. <6> 2021-08-24 10:45:02.327 +00:00 [INF] - Skipping red-iot-layer3-edge/modbus for subscription processing, as it is currently being processed. <6> 2021-08-24 10:45:02.327 +00:00 [INF] - Created cloud proxy for client red-iot-layer3-edge/modbus via AMQP, with client operation timeout 20 seconds. <6> 2021-08-24 10:45:02.327 +00:00 [INF] - Initialized cloud proxy 6a23b226-653d-4e0e-a1f4-baa6fcebdcd7 for red-iot-layer3-edge/modbus <6> 2021-08-24 10:45:02.327 +00:00 [INF] - Created cloud connection for client red-iot-layer3-edge/modbus <6> 2021-08-24 10:45:02.328 +00:00 [INF] - Processing pending subscriptions for red-iot-layer3-edge/modbus <6> 2021-08-24 10:45:02.328 +00:00 [INF] - Processing pending subscriptions for red-iot-layer3-edge/modbus <6> 2021-08-24 10:45:02.329 +00:00 [INF] - Processing pending subscriptions for red-iot-layer3-edge/modbus <6> 2021-08-24 10:45:07.224 +00:00 [INF] - Updated reported properties for red-iot-layer3-edge/$edgeHub <6> 2021-08-24 10:45:07.499 +00:00 [INF] - Done syncing reported properties for red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 10:45:59.439 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:45:59.440 +00:00 [INF] - Request finished in 0.7013ms 401 text/html <6> 2021-08-24 10:46:59.441 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:46:59.442 +00:00 [INF] - Request finished in 0.6434ms 401 text/html <6> 2021-08-24 10:47:59.441 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:47:59.442 +00:00 [INF] - Request finished in 0.6805ms 401 text/html <6> 2021-08-24 10:48:59.441 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:48:59.442 +00:00 [INF] - Request finished in 0.6266ms 401 text/html <6> 2021-08-24 10:49:59.445 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:49:59.446 +00:00 [INF] - Request finished in 6.2135ms 401 text/html <6> 2021-08-24 10:50:00.784 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 10:50:59.595 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:50:59.596 +00:00 [INF] - Request finished in 0.706ms 401 text/html <6> 2021-08-24 10:51:59.444 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:51:59.445 +00:00 [INF] - Request finished in 0.644ms 401 text/html <6> 2021-08-24 10:52:59.450 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:52:59.451 +00:00 [INF] - Request finished in 0.6901ms 401 text/html <6> 2021-08-24 10:53:59.444 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:53:59.444 +00:00 [INF] - Request finished in 0.6447ms 401 text/html <6> 2021-08-24 10:54:59.439 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:54:59.440 +00:00 [INF] - Request finished in 0.6652ms 401 text/html <6> 2021-08-24 10:55:00.783 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 10:55:59.444 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:55:59.445 +00:00 [INF] - Request finished in 0.6221ms 401 text/html <6> 2021-08-24 10:56:59.609 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:56:59.609 +00:00 [INF] - Request finished in 0.663ms 401 text/html <6> 2021-08-24 10:57:59.446 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:57:59.446 +00:00 [INF] - Request finished in 0.6307ms 401 text/html <6> 2021-08-24 10:58:59.445 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:58:59.446 +00:00 [INF] - Request finished in 0.6207ms 401 text/html <6> 2021-08-24 10:59:59.445 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 10:59:59.446 +00:00 [INF] - Request finished in 0.6247ms 401 text/html <6> 2021-08-24 11:00:00.785 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 11:00:59.445 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:00:59.446 +00:00 [INF] - Request finished in 0.6337ms 401 text/html <6> 2021-08-24 11:01:59.445 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:01:59.446 +00:00 [INF] - Request finished in 0.6375ms 401 text/html <6> 2021-08-24 11:02:59.601 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:02:59.602 +00:00 [INF] - Request finished in 0.686ms 401 text/html <6> 2021-08-24 11:03:59.448 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:03:59.449 +00:00 [INF] - Request finished in 0.6412ms 401 text/html <6> 2021-08-24 11:04:59.445 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:04:59.446 +00:00 [INF] - Request finished in 0.6192ms 401 text/html <6> 2021-08-24 11:05:00.784 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 11:05:52.716 +00:00 [INF] - Started task to cleanup processed and stale messages for endpoint iothub <6> 2021-08-24 11:05:52.718 +00:00 [INF] - Cleaned up 0 messages from queue for endpoint iothub and 0 messages from message store. <6> 2021-08-24 11:05:59.445 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:05:59.446 +00:00 [INF] - Request finished in 0.629ms 401 text/html <6> 2021-08-24 11:06:59.446 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:06:59.446 +00:00 [INF] - Request finished in 0.6356ms 401 text/html <6> 2021-08-24 11:07:59.451 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:07:59.452 +00:00 [INF] - Request finished in 0.6474ms 401 text/html <6> 2021-08-24 11:08:59.841 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:08:59.842 +00:00 [INF] - Request finished in 0.6942ms 401 text/html <6> 2021-08-24 11:09:59.446 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:09:59.447 +00:00 [INF] - Request finished in 0.6551ms 401 text/html <6> 2021-08-24 11:10:00.784 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 11:10:59.448 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:10:59.448 +00:00 [INF] - Request finished in 0.681ms 401 text/html <6> 2021-08-24 11:11:59.452 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:11:59.452 +00:00 [INF] - Request finished in 0.6415ms 401 text/html <6> 2021-08-24 11:12:59.448 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:12:59.448 +00:00 [INF] - Request finished in 0.6339ms 401 text/html <6> 2021-08-24 11:13:59.447 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:13:59.448 +00:00 [INF] - Request finished in 0.6845ms 401 text/html <6> 2021-08-24 11:14:59.611 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:14:59.611 +00:00 [INF] - Request finished in 0.6646ms 401 text/html <6> 2021-08-24 11:15:00.784 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 11:15:59.448 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:15:59.449 +00:00 [INF] - Request finished in 0.6307ms 401 text/html <6> 2021-08-24 11:16:59.445 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:16:59.446 +00:00 [INF] - Request finished in 0.6532ms 401 text/html <6> 2021-08-24 11:17:59.446 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:17:59.447 +00:00 [INF] - Request finished in 0.6037ms 401 text/html <6> 2021-08-24 11:18:59.448 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:18:59.448 +00:00 [INF] - Request finished in 0.7257ms 401 text/html <6> 2021-08-24 11:19:59.446 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:19:59.446 +00:00 [INF] - Request finished in 0.637ms 401 text/html <6> 2021-08-24 11:20:00.788 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 11:20:59.603 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:20:59.603 +00:00 [INF] - Request finished in 0.6937ms 401 text/html <6> 2021-08-24 11:21:59.445 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:21:59.446 +00:00 [INF] - Request finished in 0.6695ms 401 text/html <6> 2021-08-24 11:22:59.457 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:22:59.457 +00:00 [INF] - Request finished in 0.7089ms 401 text/html <6> 2021-08-24 11:23:59.445 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:23:59.446 +00:00 [INF] - Request finished in 0.688ms 401 text/html <6> 2021-08-24 11:24:59.445 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:24:59.446 +00:00 [INF] - Request finished in 0.6914ms 401 text/html <6> 2021-08-24 11:25:00.784 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 11:25:59.444 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:25:59.445 +00:00 [INF] - Request finished in 0.6461ms 401 text/html <6> 2021-08-24 11:26:59.602 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:26:59.603 +00:00 [INF] - Request finished in 0.6756ms 401 text/html <6> 2021-08-24 11:27:59.445 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:27:59.446 +00:00 [INF] - Request finished in 0.7242ms 401 text/html <6> 2021-08-24 11:28:59.445 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:28:59.445 +00:00 [INF] - Request finished in 0.6712ms 401 text/html <6> 2021-08-24 11:29:59.446 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:29:59.447 +00:00 [INF] - Request finished in 0.6794ms 401 text/html <6> 2021-08-24 11:30:00.786 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 11:30:59.446 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:30:59.446 +00:00 [INF] - Request finished in 0.8265ms 401 text/html <6> 2021-08-24 11:31:59.445 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:31:59.445 +00:00 [INF] - Request finished in 0.6351ms 401 text/html <6> 2021-08-24 11:32:59.597 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:32:59.597 +00:00 [INF] - Request finished in 0.698ms 401 text/html <6> 2021-08-24 11:33:59.448 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:33:59.449 +00:00 [INF] - Request finished in 0.663ms 401 text/html <6> 2021-08-24 11:34:36.818 +00:00 [INF] - Request starting HTTP/1.1 GET http://172.18.0.3:9600/metrics <6> 2021-08-24 11:34:36.823 +00:00 [INF] - Request finished in 5.3778ms 200 text/plain; version=0.0.4; charset=utf-8 <6> 2021-08-24 11:34:51.955 +00:00 [INF] - Starting compaction of stores <6> 2021-08-24 11:34:51.956 +00:00 [INF] - Starting compaction of store EdgeTwin <6> 2021-08-24 11:34:51.960 +00:00 [INF] - Starting compaction of store default <6> 2021-08-24 11:34:51.960 +00:00 [INF] - Starting compaction of store sessions <6> 2021-08-24 11:34:51.960 +00:00 [INF] - Starting compaction of store checkpoints <6> 2021-08-24 11:34:51.960 +00:00 [INF] - Starting compaction of store messages <6> 2021-08-24 11:34:51.961 +00:00 [INF] - Starting compaction of store iothub <6> 2021-08-24 11:34:51.961 +00:00 [INF] - Starting compaction of store twins <6> 2021-08-24 11:34:51.961 +00:00 [INF] - Starting compaction of store DeviceScopeCache <6> 2021-08-24 11:34:51.961 +00:00 [INF] - Starting compaction of store MetadataStore <6> 2021-08-24 11:34:56.552 +00:00 [INF] - Starting refresh of device scope identities cache <6> 2021-08-24 11:34:59.436 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:34:59.436 +00:00 [INF] - Request finished in 0.6186ms 401 text/html <6> 2021-08-24 11:35:00.784 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 11:35:01.413 +00:00 [INF] - Starting periodic operation Get EdgeHub config... <6> 2021-08-24 11:35:01.881 +00:00 [INF] - Obtained edge hub config from module twin <6> 2021-08-24 11:35:01.891 +00:00 [INF] - Set the following 1 route(s) in edge hub <6> 2021-08-24 11:35:01.891 +00:00 [INF] - route: FROM /messages/* INTO $upstream <6> 2021-08-24 11:35:01.891 +00:00 [INF] - Updated message store TTL to 7200 seconds <6> 2021-08-24 11:35:01.892 +00:00 [INF] - Updated the edge hub store and forward configuration <6> 2021-08-24 11:35:01.892 +00:00 [INF] - Successfully completed periodic operation Get EdgeHub config <6> 2021-08-24 11:35:02.161 +00:00 [INF] - Updated reported properties for red-iot-layer3-edge/$edgeHub <6> 2021-08-24 11:35:59.449 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:35:59.450 +00:00 [INF] - Request finished in 0.8762ms 401 text/html <6> 2021-08-24 11:36:22.718 +00:00 [INF] - Started task to cleanup processed and stale messages for endpoint iothub <6> 2021-08-24 11:36:22.719 +00:00 [INF] - Cleaned up 0 messages from queue for endpoint iothub and 0 messages from message store. <6> 2021-08-24 11:36:59.445 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:36:59.445 +00:00 [INF] - Request finished in 0.6197ms 401 text/html <6> 2021-08-24 11:37:59.458 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:37:59.458 +00:00 [INF] - Request finished in 0.6267ms 401 text/html <6> 2021-08-24 11:38:59.596 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:38:59.597 +00:00 [INF] - Request finished in 0.6726ms 401 text/html <6> 2021-08-24 11:39:59.656 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:39:59.657 +00:00 [INF] - Request finished in 0.7671ms 401 text/html <6> 2021-08-24 11:40:00.783 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 11:40:59.447 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:40:59.448 +00:00 [INF] - Request finished in 0.7214ms 401 text/html <6> 2021-08-24 11:41:59.449 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:41:59.450 +00:00 [INF] - Request finished in 0.6737ms 401 text/html <6> 2021-08-24 11:42:59.442 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:42:59.443 +00:00 [INF] - Request finished in 0.622ms 401 text/html <6> 2021-08-24 11:43:59.448 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:43:59.449 +00:00 [INF] - Request finished in 0.6457ms 401 text/html <6> 2021-08-24 11:44:59.597 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:44:59.597 +00:00 [INF] - Request finished in 0.7544ms 401 text/html <6> 2021-08-24 11:45:00.783 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 11:45:02.326 +00:00 [INF] - Closing cloud proxy 6a23b226-653d-4e0e-a1f4-baa6fcebdcd7 for red-iot-layer3-edge/modbus because of inactivity <6> 2021-08-24 11:45:02.329 +00:00 [INF] - Closing receiver in cloud proxy 6a23b226-653d-4e0e-a1f4-baa6fcebdcd7 for red-iot-layer3-edge/modbus <6> 2021-08-24 11:45:02.329 +00:00 [INF] - Closed cloud proxy 6a23b226-653d-4e0e-a1f4-baa6fcebdcd7 for red-iot-layer3-edge/modbus <6> 2021-08-24 11:45:59.444 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:45:59.445 +00:00 [INF] - Request finished in 0.6736ms 401 text/html <6> 2021-08-24 11:46:59.444 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:46:59.445 +00:00 [INF] - Request finished in 0.6312ms 401 text/html <6> 2021-08-24 11:47:59.444 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:47:59.445 +00:00 [INF] - Request finished in 0.6432ms 401 text/html <6> 2021-08-24 11:48:59.443 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:48:59.444 +00:00 [INF] - Request finished in 0.6812ms 401 text/html <6> 2021-08-24 11:49:59.444 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:49:59.444 +00:00 [INF] - Request finished in 0.628ms 401 text/html <6> 2021-08-24 11:50:00.785 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 11:50:00.785 +00:00 [INF] - Error authenticating token for red-iot-layer3-edge/IoTEdgeAPIProxy because the token is expired or could not be parsed <6> 2021-08-24 11:50:00.786 +00:00 [INF] - Credentials for client red-iot-layer3-edge/IoTEdgeAPIProxy are not valid. <4> 2021-08-24 11:50:00.786 +00:00 [WRN] - Reauthenticating client red-iot-layer3-edge/IoTEdgeAPIProxy failed, removing client connection <6> 2021-08-24 11:50:00.786 +00:00 [INF] - Unable to re-authenticate red-iot-layer3-edge/IoTEdgeAPIProxy, dropping client connection. <6> 2021-08-24 11:50:00.786 +00:00 [INF] - Removing device connection for device red-iot-layer3-edge/IoTEdgeAPIProxy with removeCloudConnection flag 'True'. <4> 2021-08-24 11:50:00.787 +00:00 [WRN] - Closing connection for device: red-iot-layer3-edge/IoTEdgeAPIProxy, Microsoft.Azure.Devices.Edge.Hub.Core.EdgeHubConnectionException: Connection closed for device red-iot-layer3-edge/IoTEdgeAPIProxy., <6> 2021-08-24 11:50:00.787 +00:00 [INF] - Disposing MessagingServiceClient for device Id red-iot-layer3-edge/IoTEdgeAPIProxy because of exception - Microsoft.Azure.Devices.Edge.Hub.Core.EdgeHubConnectionException: Connection closed for device red-iot-layer3-edge/IoTEdgeAPIProxy. <6> 2021-08-24 11:50:00.787 +00:00 [INF] - Closing device proxy for device Id red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 11:50:00.796 +00:00 [INF] - Closing receiver in cloud proxy 18aa625d-f72c-4f8e-a35c-69d0f027fdaf for red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 11:50:00.797 +00:00 [INF] - Closed cloud proxy 18aa625d-f72c-4f8e-a35c-69d0f027fdaf for red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 11:50:00.797 +00:00 [INF] - Device connection removed for device red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 11:50:01.354 +00:00 [INF] - Client red-iot-layer3-edge/IoTEdgeAPIProxy in device scope authenticated locally. <6> 2021-08-24 11:50:01.355 +00:00 [INF] - Successfully generated identity for clientId red-iot-layer3-edge/IoTEdgeAPIProxy and username 192.168.1.90/red-iot-layer3-edge/IoTEdgeAPIProxy/?api-version=2018-06-30 <6> 2021-08-24 11:50:01.355 +00:00 [INF] - ClientAuthenticated, red-iot-layer3-edge/IoTEdgeAPIProxy, 0af531bf <6> 2021-08-24 11:50:01.359 +00:00 [INF] - New device connection for device red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 11:50:01.359 +00:00 [INF] - Client red-iot-layer3-edge/IoTEdgeAPIProxy connected to edgeHub, processing existing subscriptions. <6> 2021-08-24 11:50:01.362 +00:00 [INF] - Bind device proxy for device red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 11:50:01.362 +00:00 [INF] - Binding message channel for device Id red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 11:50:01.365 +00:00 [INF] - Attempting to connect to IoT Hub for client red-iot-layer3-edge/IoTEdgeAPIProxy via AMQP... <6> 2021-08-24 11:50:01.372 +00:00 [INF] - Processing pending subscriptions for red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 11:50:01.413 +00:00 [INF] - Updated reported properties for red-iot-layer3-edge/$edgeHub <6> 2021-08-24 11:50:01.685 +00:00 [INF] - Cloud connection for red-iot-layer3-edge/IoTEdgeAPIProxy is True <6> 2021-08-24 11:50:01.686 +00:00 [INF] - Connection status for red-iot-layer3-edge/IoTEdgeAPIProxy changed to ConnectionEstablished <6> 2021-08-24 11:50:01.686 +00:00 [INF] - Client red-iot-layer3-edge/IoTEdgeAPIProxy connected to cloud, processing existing subscriptions. <6> 2021-08-24 11:50:01.687 +00:00 [INF] - Skipping red-iot-layer3-edge/IoTEdgeAPIProxy for subscription processing, as it is currently being processed. <6> 2021-08-24 11:50:01.687 +00:00 [INF] - Created cloud proxy for client red-iot-layer3-edge/IoTEdgeAPIProxy via AMQP, with client operation timeout 20 seconds. <6> 2021-08-24 11:50:01.687 +00:00 [INF] - Initialized cloud proxy d3f79f38-fdf5-4486-816c-7a58d71a93c5 for red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 11:50:01.688 +00:00 [INF] - Created cloud connection for client red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 11:50:01.738 +00:00 [INF] - Processing pending subscriptions for red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 11:50:01.739 +00:00 [INF] - Processing pending subscriptions for red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 11:50:07.059 +00:00 [INF] - Updated reported properties for red-iot-layer3-edge/$edgeHub <6> 2021-08-24 11:50:07.323 +00:00 [INF] - Done syncing reported properties for red-iot-layer3-edge/IoTEdgeAPIProxy <6> 2021-08-24 11:50:59.444 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:50:59.444 +00:00 [INF] - Request finished in 0.6604ms 401 text/html <6> 2021-08-24 11:51:59.443 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:51:59.444 +00:00 [INF] - Request finished in 0.6762ms 401 text/html <6> 2021-08-24 11:52:59.449 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:52:59.449 +00:00 [INF] - Request finished in 0.6345ms 401 text/html <6> 2021-08-24 11:53:59.444 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:53:59.445 +00:00 [INF] - Request finished in 0.6374ms 401 text/html <6> 2021-08-24 11:54:59.443 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:54:59.444 +00:00 [INF] - Request finished in 0.7055ms 401 text/html <6> 2021-08-24 11:55:00.785 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 11:55:00.786 +00:00 [INF] - Error authenticating token for red-iot-layer3-edge/modbus because the token is expired or could not be parsed <6> 2021-08-24 11:55:00.787 +00:00 [INF] - Credentials for client red-iot-layer3-edge/modbus are not valid. <4> 2021-08-24 11:55:00.787 +00:00 [WRN] - Reauthenticating client red-iot-layer3-edge/modbus failed, removing client connection <6> 2021-08-24 11:55:00.787 +00:00 [INF] - Unable to re-authenticate red-iot-layer3-edge/modbus, dropping client connection. <6> 2021-08-24 11:55:00.787 +00:00 [INF] - Removing device connection for device red-iot-layer3-edge/modbus with removeCloudConnection flag 'True'. <4> 2021-08-24 11:55:00.788 +00:00 [WRN] - Closing connection for device: red-iot-layer3-edge/modbus, Microsoft.Azure.Devices.Edge.Hub.Core.EdgeHubConnectionException: Connection closed for device red-iot-layer3-edge/modbus., <6> 2021-08-24 11:55:00.788 +00:00 [INF] - Disposing MessagingServiceClient for device Id red-iot-layer3-edge/modbus because of exception - Microsoft.Azure.Devices.Edge.Hub.Core.EdgeHubConnectionException: Connection closed for device red-iot-layer3-edge/modbus. <6> 2021-08-24 11:55:00.788 +00:00 [INF] - Closing device proxy for device Id red-iot-layer3-edge/modbus <6> 2021-08-24 11:55:00.789 +00:00 [INF] - Device connection removed for device red-iot-layer3-edge/modbus <6> 2021-08-24 11:55:01.344 +00:00 [INF] - Client red-iot-layer3-edge/modbus in device scope authenticated locally. <6> 2021-08-24 11:55:01.345 +00:00 [INF] - Successfully generated identity for clientId red-iot-layer3-edge/modbus and username 192.168.1.90/red-iot-layer3-edge/modbus/?api-version=2020-09-30&DeviceClientType=.NET%2F1.37.2%20%28.NET%20Core%203.1.17%3B%20Linux%205.4.24-iot-gate-imx8-2.4%2Bgbabac008e5cf%20%231%20SMP%20PREEMPT%20Tue%20Mar%202%2012%3A34%3A06%20UTC%202021%3B%20Arm64%29 <6> 2021-08-24 11:55:01.345 +00:00 [INF] - ClientAuthenticated, red-iot-layer3-edge/modbus, 2726fffb <6> 2021-08-24 11:55:01.348 +00:00 [INF] - New device connection for device red-iot-layer3-edge/modbus <6> 2021-08-24 11:55:01.348 +00:00 [INF] - Client red-iot-layer3-edge/modbus connected to edgeHub, processing existing subscriptions. <6> 2021-08-24 11:55:01.357 +00:00 [INF] - Bind device proxy for device red-iot-layer3-edge/modbus <6> 2021-08-24 11:55:01.358 +00:00 [INF] - Binding message channel for device Id red-iot-layer3-edge/modbus <6> 2021-08-24 11:55:01.360 +00:00 [INF] - Attempting to connect to IoT Hub for client red-iot-layer3-edge/modbus via AMQP... <6> 2021-08-24 11:55:01.361 +00:00 [INF] - Processing pending subscriptions for red-iot-layer3-edge/modbus <6> 2021-08-24 11:55:01.369 +00:00 [INF] - Processing subscriptions TwinResponse, ModuleMessages for client red-iot-layer3-edge/modbus. <6> 2021-08-24 11:55:01.371 +00:00 [INF] - Set subscriptions from session state for red-iot-layer3-edge/modbus <6> 2021-08-24 11:55:01.379 +00:00 [INF] - Processing subscriptions TwinResponse, ModuleMessages for client red-iot-layer3-edge/modbus. <6> 2021-08-24 11:55:01.381 +00:00 [INF] - Set subscriptions from session state for red-iot-layer3-edge/modbus <6> 2021-08-24 11:55:01.453 +00:00 [INF] - Updated reported properties for red-iot-layer3-edge/$edgeHub <6> 2021-08-24 11:55:01.820 +00:00 [INF] - Cloud connection for red-iot-layer3-edge/modbus is True <6> 2021-08-24 11:55:01.821 +00:00 [INF] - Connection status for red-iot-layer3-edge/modbus changed to ConnectionEstablished <6> 2021-08-24 11:55:01.821 +00:00 [INF] - Client red-iot-layer3-edge/modbus connected to cloud, processing existing subscriptions. <6> 2021-08-24 11:55:01.821 +00:00 [INF] - Skipping red-iot-layer3-edge/modbus for subscription processing, as it is currently being processed. <6> 2021-08-24 11:55:01.821 +00:00 [INF] - Created cloud proxy for client red-iot-layer3-edge/modbus via AMQP, with client operation timeout 20 seconds. <6> 2021-08-24 11:55:01.822 +00:00 [INF] - Initialized cloud proxy e6ea2212-7a2a-41b6-9d2c-c2c7b5f37eb0 for red-iot-layer3-edge/modbus <6> 2021-08-24 11:55:01.822 +00:00 [INF] - Created cloud connection for client red-iot-layer3-edge/modbus <6> 2021-08-24 11:55:01.823 +00:00 [INF] - Processing pending subscriptions for red-iot-layer3-edge/modbus <6> 2021-08-24 11:55:01.823 +00:00 [INF] - Processing pending subscriptions for red-iot-layer3-edge/modbus <6> 2021-08-24 11:55:01.824 +00:00 [INF] - Processing pending subscriptions for red-iot-layer3-edge/modbus <6> 2021-08-24 11:55:07.046 +00:00 [INF] - Updated reported properties for red-iot-layer3-edge/$edgeHub <6> 2021-08-24 11:55:59.595 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:55:59.596 +00:00 [INF] - Request finished in 0.7327ms 401 text/html <6> 2021-08-24 11:56:59.444 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:56:59.444 +00:00 [INF] - Request finished in 0.6436ms 401 text/html <6> 2021-08-24 11:57:59.442 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:57:59.443 +00:00 [INF] - Request finished in 0.7232ms 401 text/html <6> 2021-08-24 11:58:59.444 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:58:59.445 +00:00 [INF] - Request finished in 0.7195ms 401 text/html <6> 2021-08-24 11:59:59.443 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 11:59:59.443 +00:00 [INF] - Request finished in 0.6697ms 401 text/html <6> 2021-08-24 12:00:00.785 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 12:00:59.442 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:00:59.443 +00:00 [INF] - Request finished in 0.6709ms 401 text/html <6> 2021-08-24 12:01:59.593 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:01:59.594 +00:00 [INF] - Request finished in 0.6899ms 401 text/html <6> 2021-08-24 12:02:59.449 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:02:59.450 +00:00 [INF] - Request finished in 6.1066ms 401 text/html <6> 2021-08-24 12:03:59.444 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:03:59.444 +00:00 [INF] - Request finished in 0.6527ms 401 text/html <6> 2021-08-24 12:04:59.444 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:04:59.444 +00:00 [INF] - Request finished in 0.6336ms 401 text/html <6> 2021-08-24 12:05:00.784 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 12:05:59.442 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:05:59.443 +00:00 [INF] - Request finished in 0.619ms 401 text/html <6> 2021-08-24 12:06:52.720 +00:00 [INF] - Started task to cleanup processed and stale messages for endpoint iothub <6> 2021-08-24 12:06:52.721 +00:00 [INF] - Cleaned up 0 messages from queue for endpoint iothub and 0 messages from message store. <6> 2021-08-24 12:06:59.444 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:06:59.444 +00:00 [INF] - Request finished in 0.6319ms 401 text/html <6> 2021-08-24 12:07:59.601 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:07:59.601 +00:00 [INF] - Request finished in 0.7102ms 401 text/html <6> 2021-08-24 12:08:59.444 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:08:59.444 +00:00 [INF] - Request finished in 0.6951ms 401 text/html <6> 2021-08-24 12:09:59.449 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:09:59.450 +00:00 [INF] - Request finished in 0.8876ms 401 text/html <6> 2021-08-24 12:10:00.785 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 12:10:59.680 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:10:59.681 +00:00 [INF] - Request finished in 0.6087ms 401 text/html <6> 2021-08-24 12:11:59.444 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:11:59.444 +00:00 [INF] - Request finished in 0.6504ms 401 text/html <6> 2021-08-24 12:12:59.441 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:12:59.442 +00:00 [INF] - Request finished in 0.6499ms 401 text/html <6> 2021-08-24 12:13:59.599 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:13:59.600 +00:00 [INF] - Request finished in 0.6882ms 401 text/html <6> 2021-08-24 12:14:59.444 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:14:59.444 +00:00 [INF] - Request finished in 0.6561ms 401 text/html <6> 2021-08-24 12:15:00.782 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 12:15:59.448 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:15:59.449 +00:00 [INF] - Request finished in 0.7037ms 401 text/html <6> 2021-08-24 12:16:59.444 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:16:59.445 +00:00 [INF] - Request finished in 0.6274ms 401 text/html <6> 2021-08-24 12:17:59.442 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:17:59.443 +00:00 [INF] - Request finished in 0.6215ms 401 text/html <6> 2021-08-24 12:18:59.443 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:18:59.444 +00:00 [INF] - Request finished in 0.6417ms 401 text/html <6> 2021-08-24 12:19:59.603 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:19:59.603 +00:00 [INF] - Request finished in 0.7025ms 401 text/html <6> 2021-08-24 12:20:00.788 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 12:20:59.444 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:20:59.445 +00:00 [INF] - Request finished in 0.6525ms 401 text/html <6> 2021-08-24 12:21:59.483 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:21:59.484 +00:00 [INF] - Request finished in 0.6461ms 401 text/html <6> 2021-08-24 12:22:59.452 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:22:59.452 +00:00 [INF] - Request finished in 0.6625ms 401 text/html <6> 2021-08-24 12:23:59.442 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:23:59.443 +00:00 [INF] - Request finished in 0.6419ms 401 text/html <6> 2021-08-24 12:24:59.442 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:24:59.443 +00:00 [INF] - Request finished in 0.641ms 401 text/html <6> 2021-08-24 12:25:00.784 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 12:25:59.604 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:25:59.605 +00:00 [INF] - Request finished in 0.6891ms 401 text/html <6> 2021-08-24 12:26:59.444 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:26:59.445 +00:00 [INF] - Request finished in 0.7012ms 401 text/html <6> 2021-08-24 12:27:59.441 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:27:59.442 +00:00 [INF] - Request finished in 0.7554ms 401 text/html <6> 2021-08-24 12:28:59.442 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:28:59.442 +00:00 [INF] - Request finished in 0.6702ms 401 text/html <6> 2021-08-24 12:29:59.442 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:29:59.442 +00:00 [INF] - Request finished in 0.6456ms 401 text/html <6> 2021-08-24 12:30:00.785 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 12:30:59.443 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:30:59.444 +00:00 [INF] - Request finished in 0.7065ms 401 text/html <6> 2021-08-24 12:31:59.594 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:31:59.594 +00:00 [INF] - Request finished in 0.6849ms 401 text/html <6> 2021-08-24 12:32:59.444 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:32:59.444 +00:00 [INF] - Request finished in 0.648ms 401 text/html <6> 2021-08-24 12:33:59.447 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:33:59.448 +00:00 [INF] - Request finished in 0.7422ms 401 text/html <6> 2021-08-24 12:34:36.924 +00:00 [INF] - Request starting HTTP/1.1 GET http://172.18.0.3:9600/metrics <6> 2021-08-24 12:34:36.929 +00:00 [INF] - Request finished in 4.3865ms 200 text/plain; version=0.0.4; charset=utf-8 <6> 2021-08-24 12:34:57.684 +00:00 [INF] - Starting refresh of device scope identities cache <6> 2021-08-24 12:34:59.435 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:34:59.435 +00:00 [INF] - Request finished in 0.7155ms 401 text/html <6> 2021-08-24 12:35:00.783 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 12:35:01.891 +00:00 [INF] - Starting periodic operation Get EdgeHub config... <6> 2021-08-24 12:35:02.337 +00:00 [INF] - Obtained edge hub config from module twin <6> 2021-08-24 12:35:02.345 +00:00 [INF] - Set the following 1 route(s) in edge hub <6> 2021-08-24 12:35:02.346 +00:00 [INF] - route: FROM /messages/* INTO $upstream <6> 2021-08-24 12:35:02.346 +00:00 [INF] - Updated message store TTL to 7200 seconds <6> 2021-08-24 12:35:02.346 +00:00 [INF] - Updated the edge hub store and forward configuration <6> 2021-08-24 12:35:02.346 +00:00 [INF] - Successfully completed periodic operation Get EdgeHub config <6> 2021-08-24 12:35:02.612 +00:00 [INF] - Updated reported properties for red-iot-layer3-edge/$edgeHub <6> 2021-08-24 12:35:59.442 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:35:59.443 +00:00 [INF] - Request finished in 0.6617ms 401 text/html <6> 2021-08-24 12:36:59.443 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:36:59.444 +00:00 [INF] - Request finished in 0.6825ms 401 text/html <6> 2021-08-24 12:37:22.731 +00:00 [INF] - Started task to cleanup processed and stale messages for endpoint iothub <6> 2021-08-24 12:37:22.732 +00:00 [INF] - Cleaned up 0 messages from queue for endpoint iothub and 0 messages from message store. <6> 2021-08-24 12:37:59.604 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:37:59.604 +00:00 [INF] - Request finished in 0.6892ms 401 text/html <6> 2021-08-24 12:38:59.442 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:38:59.442 +00:00 [INF] - Request finished in 0.6329ms 401 text/html <6> 2021-08-24 12:39:59.442 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:39:59.442 +00:00 [INF] - Request finished in 0.6282ms 401 text/html <6> 2021-08-24 12:40:00.785 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 12:40:59.442 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:40:59.442 +00:00 [INF] - Request finished in 0.6661ms 401 text/html <6> 2021-08-24 12:41:59.640 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:41:59.640 +00:00 [INF] - Request finished in 0.6454ms 401 text/html <6> 2021-08-24 12:42:59.445 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:42:59.446 +00:00 [INF] - Request finished in 0.6822ms 401 text/html <6> 2021-08-24 12:43:59.597 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:43:59.598 +00:00 [INF] - Request finished in 0.743ms 401 text/html <6> 2021-08-24 12:44:59.442 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:44:59.443 +00:00 [INF] - Request finished in 0.634ms 401 text/html <6> 2021-08-24 12:45:00.785 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 12:45:59.442 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:45:59.443 +00:00 [INF] - Request finished in 0.6331ms 401 text/html <6> 2021-08-24 12:46:59.444 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:46:59.444 +00:00 [INF] - Request finished in 0.6405ms 401 text/html <6> 2021-08-24 12:47:59.447 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:47:59.448 +00:00 [INF] - Request finished in 0.6876ms 401 text/html <6> 2021-08-24 12:48:59.442 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:48:59.442 +00:00 [INF] - Request finished in 0.6461ms 401 text/html <6> 2021-08-24 12:49:59.598 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:49:59.599 +00:00 [INF] - Request finished in 0.6707ms 401 text/html <6> 2021-08-24 12:50:00.785 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <6> 2021-08-24 12:50:59.441 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:50:59.442 +00:00 [INF] - Request finished in 0.6897ms 401 text/html <6> 2021-08-24 12:51:46.216 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/0000/modules <6> 2021-08-24 12:51:46.216 +00:00 [INF] - Request finished in 0.6212ms 401 text/html <4> 2021-08-24 12:51:47.920 +00:00 [WRN] - TLS handshake failed., System.IO.IOException: Channel is closed, 6776ee91 <6> 2021-08-24 12:51:53.540 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/0000/modules <6> 2021-08-24 12:51:53.541 +00:00 [INF] - Request finished in 0.6195ms 401 text/html <4> 2021-08-24 12:51:55.494 +00:00 [WRN] - TLS handshake failed., System.IO.IOException: Channel is closed, 1accad6c <6> 2021-08-24 12:51:59.434 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:51:59.434 +00:00 [INF] - Request finished in 0.6685ms 401 text/html <6> 2021-08-24 12:52:59.441 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:52:59.442 +00:00 [INF] - Request finished in 0.647ms 401 text/html <6> 2021-08-24 12:53:59.441 +00:00 [INF] - Request starting HTTP/1.1 GET https://edgehub/devices/red-iot-layer2-edge/modules/$edgeAgent?api-version=2017-11-08-preview <6> 2021-08-24 12:53:59.442 +00:00 [INF] - Request finished in 0.8461ms 401 text/html ```

huguesBouvier commented 2 years ago

Thanks! I think I understand what is happening. You are using nested edge with an older edgeHub version. edge 1.1 doesn't support nested edge.

The iotedge check (response to 443) show edgeHub is not sending back the correct header. Additionally, the edgeHub logs shows that it is not understanding the request from the child edgeHub.

Try updating edgeHub 1.2, it should fix the issue.

patrikSohlman commented 2 years ago

Thanks @huguesBouvier, changing version to 1.2 did the trick. Maybe that should be added to the guide as that one is specific in what edgeAgent version to use, not the edgeHub. I am guessing that they mirror their versioning, but I think a mention would help in clarifiying that. Again thanks, you are a lifesaver.

huguesBouvier commented 2 years ago

You're welcome! I will close the issue. Let us know if you have more problems.

Azure / iotedge