search

Azure / iotedge

The IoT Edge OSS project
MIT License
1.46k stars 459 forks source link

SimulatedTemperatureSensor should start in offline mode after a reboot of the Edge device #5130

Closed shaeussler closed 3 years ago

shaeussler commented 3 years ago

Expected Behavior

In offline mode after a reboot of the Edge device the SimulatedTemperatureSensor should start and send data.

Current Behavior

SimulatedTemperatureSensor hangs after module startup

Steps to Reproduce

  1. Install IoT Edge on an Azure Ubuntu VM
  2. Add SimulatedTemperatureSensor as a module
  3. Set Azure Ubuntu VM offline using the method from https://microsoftlearning.github.io/AZ-220-Microsoft-Azure-IoT-Developer/Instructions/Labs/LAB_AK_14-iot-edge-device-in-restricted-network.html#exercise-8-test-device-connectivity-and-offline-support
  4. Reboot the Edge device
  5. iotedge logs -f SimulatedTemperatureSensor

Context (Environment)

Output of iotedge check

I run the iotedge check in online mode

Click here ``` Configuration checks (aziot-identity-service) --------------------------------------------- √ keyd configuration is well-formed - OK √ certd configuration is well-formed - OK √ tpmd configuration is well-formed - OK √ identityd configuration is well-formed - OK √ daemon configurations up-to-date with config.toml - OK √ identityd config toml file specifies a valid hostname - OK √ aziot-identity-service package is up-to-date - OK √ host time is close to reference time - OK √ preloaded certificates are valid - OK √ keyd is running - OK √ certd is running - OK √ identityd is running - OK √ read all preloaded certificates from the Certificates Service - OK √ read all preloaded key pairs from the Keys Service - OK √ ensure all preloaded certificates match preloaded private keys with the same ID - OK Connectivity checks (aziot-identity-service) -------------------------------------------- √ host can connect to and perform TLS handshake with iothub AMQP port - OK √ host can connect to and perform TLS handshake with iothub HTTPS / WebSockets port - OK √ host can connect to and perform TLS handshake with iothub MQTT port - OK Configuration checks -------------------- √ aziot-edged configuration is well-formed - OK √ configuration up-to-date with config.toml - OK √ container engine is installed and functional - OK √ configuration has correct URIs for daemon mgmt endpoint - OK √ aziot-edge package is up-to-date - OK √ container time is close to host time - OK √ DNS server - OK √ production readiness: container engine - OK ‼ production readiness: logs policy - Warning Container engine is not configured to rotate module logs which may cause it run out of disk space. Please see https://aka.ms/iotedge-prod-checklist-logs for best practices. You can ignore this warning if you are setting log policy per module in the Edge deployment. ‼ production readiness: Edge Agent's storage directory is persisted on the host filesystem - Warning The edgeAgent module is not configured to persist its /tmp/edgeAgent directory on the host filesystem. Data might be lost if the module is deleted or updated. Please see https://aka.ms/iotedge-storage-host for best practices. ‼ production readiness: Edge Hub's storage directory is persisted on the host filesystem - Warning The edgeHub module is not configured to persist its /tmp/edgeHub directory on the host filesystem. Data might be lost if the module is deleted or updated. Please see https://aka.ms/iotedge-storage-host for best practices. √ Agent image is valid and can be pulled from upstream - OK Connectivity checks ------------------- √ container on the default network can connect to upstream AMQP port - OK √ container on the default network can connect to upstream HTTPS / WebSockets port - OK √ container on the default network can connect to upstream MQTT port - OK √ container on the IoT Edge module network can connect to upstream AMQP port - OK √ container on the IoT Edge module network can connect to upstream HTTPS / WebSockets port - OK √ container on the IoT Edge module network can connect to upstream MQTT port - OK 33 check(s) succeeded. 3 check(s) raised warnings. Re-run with --verbose for more details. ```

Device Information

Runtime Versions

Server: Engine: Version: 20.10.6+azure API version: 1.41 (minimum version 1.12) Go version: go1.13.15 Git commit: 8728dd246c3ab53105434eef8ffe997b6fd14dc6 Built: Fri Apr 9 22:06:18 2021 OS/Arch: linux/amd64 Experimental: false containerd: Version: 1.4.6+azure GitCommit: d71fcd7d8303cbf684402823e425e9dd2e99285d runc: Version: 1.0.0-rc95 GitCommit: b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7 docker-init: Version: 0.19.0 GitCommit:

Logs

aziot-edged logs ``` -- Reboot -- Jun 18 15:53:21 di-10-ubuntu1804 systemd[1]: Started Azure IoT Edge daemon. Jun 18 15:53:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:21Z [INFO] - Starting Azure IoT Edge Module Runtime Jun 18 15:53:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:21Z [INFO] - Version - 1.2.1 Jun 18 15:53:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:21Z [INFO] - Initializing the module runtime... Jun 18 15:53:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:21Z [INFO] - Initializing module runtime... Jun 18 15:53:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:21Z [INFO] - Using runtime network id azure-iot-edge Jun 18 15:53:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:21Z [INFO] - Successfully initialized module runtime Jun 18 15:53:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:21Z [INFO] - Finished initializing the module runtime. Jun 18 15:53:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:21Z [INFO] - Obtaining edge device provisioning data... Jun 18 15:53:21 di-10-ubuntu1804 systemd[1]: Started Azure IoT Identity Service. Jun 18 15:53:21 di-10-ubuntu1804 aziot-identityd[1716]: 2021-06-18T15:53:21Z [INFO] - Starting service... Jun 18 15:53:21 di-10-ubuntu1804 aziot-identityd[1716]: 2021-06-18T15:53:21Z [INFO] - Version - 1.2.0 Jun 18 15:53:21 di-10-ubuntu1804 aziot-identityd[1716]: 2021-06-18T15:53:21Z [INFO] - Provisioning starting. Reason: Startup Jun 18 15:53:21 di-10-ubuntu1804 aziot-identityd[1716]: 2021-06-18T15:53:21Z [INFO] - Provisioning complete. Jun 18 15:53:21 di-10-ubuntu1804 aziot-identityd[1716]: 2021-06-18T15:53:21Z [INFO] - Identity reconciliation started. Reason: Startup Jun 18 15:53:21 di-10-ubuntu1804 aziot-identityd[1716]: 2021-06-18T15:53:21Z [INFO] - Identity reconciliation complete. Jun 18 15:53:21 di-10-ubuntu1804 aziot-identityd[1716]: 2021-06-18T15:53:21Z [INFO] - Starting server... Jun 18 15:53:21 di-10-ubuntu1804 aziot-identityd[1716]: 2021-06-18T15:53:21Z [INFO] - <-- POST /identities/device?api-version=2020-09-01 {"content-type": "application/json", "host": "2f72756e2f617a696f742f6964656e74697479642e736f636b:0", "content-length": "16"} Jun 18 15:53:21 di-10-ubuntu1804 systemd[1]: Started Azure IoT Keys Service. Jun 18 15:53:21 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:21Z [INFO] - Starting service... Jun 18 15:53:21 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:21Z [INFO] - Version - 1.2.0 Jun 18 15:53:21 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:21Z [INFO] - Loaded libaziot-keys with version 0x02000000 Jun 18 15:53:21 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:21Z [INFO] - Starting server... Jun 18 15:53:21 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:21Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"} Jun 18 15:53:21 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:21Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:21 di-10-ubuntu1804 aziot-identityd[1716]: 2021-06-18T15:53:21Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:21Z [INFO] - Finished provisioning edge device. Jun 18 15:53:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:21Z [INFO] - Stopping all modules... Jun 18 15:53:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:21Z [INFO] - Stopping module SimulatedTemperatureSensor... Jun 18 15:53:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:21Z [INFO] - Stopping module edgeHub... Jun 18 15:53:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:21Z [INFO] - Stopping module edgeAgent... Jun 18 15:53:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:21Z [WARN] - Could not stop module SimulatedTemperatureSensor Jun 18 15:53:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:21Z [WARN] - caused by: Target of operation already in this state Jun 18 15:53:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:21Z [WARN] - Could not stop module edgeHub Jun 18 15:53:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:21Z [WARN] - caused by: Target of operation already in this state Jun 18 15:53:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:21Z [WARN] - Could not stop module edgeAgent Jun 18 15:53:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:21Z [WARN] - caused by: Target of operation already in this state Jun 18 15:53:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:21Z [INFO] - Finished stopping modules. Jun 18 15:53:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:21Z [INFO] - Detecting if device information has changed... Jun 18 15:53:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:21Z [INFO] - Starting management API... Jun 18 15:53:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:21Z [INFO] - Starting workload API... Jun 18 15:53:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:21Z [INFO] - Starting watchdog with 60 second frequency... Jun 18 15:53:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:21Z [INFO] - Listening on fd://aziot-edged.mgmt.socket with 1 thread for management API. Jun 18 15:53:21 di-10-ubuntu1804 systemd[1]: Started Azure IoT Certificates Service. Jun 18 15:53:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:21Z [INFO] - Checking edge runtime status Jun 18 15:53:21 di-10-ubuntu1804 aziot-certd[1768]: 2021-06-18T15:53:21Z [INFO] - Starting service... Jun 18 15:53:21 di-10-ubuntu1804 aziot-certd[1768]: 2021-06-18T15:53:21Z [INFO] - Version - 1.2.0 Jun 18 15:53:21 di-10-ubuntu1804 aziot-certd[1768]: 2021-06-18T15:53:21Z [INFO] - Starting server... Jun 18 15:53:21 di-10-ubuntu1804 aziot-certd[1768]: 2021-06-18T15:53:21Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"} Jun 18 15:53:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:21Z [INFO] - Edge runtime status is failed, starting module now... Jun 18 15:53:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:21Z [INFO] - Starting module edgeAgent... Jun 18 15:53:21 di-10-ubuntu1804 aziot-certd[1768]: 2021-06-18T15:53:21Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:21Z [INFO] - Listening on fd://aziot-edged.workload.socket with 1 thread for workload API. Jun 18 15:53:22 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:22Z [INFO] - Successfully started module edgeAgent Jun 18 15:53:23 di-10-ubuntu1804 aziot-certd[1768]: 2021-06-18T15:53:23Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"} Jun 18 15:53:23 di-10-ubuntu1804 aziot-certd[1768]: 2021-06-18T15:53:23Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:23 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:23Z [INFO] - [work] - - - [2021-06-18 15:53:23.380650900 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 8024 "-" "-" auth_id(-) Jun 18 15:53:23 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:23Z [INFO] - Querying system info... Jun 18 15:53:23 di-10-ubuntu1804 aziot-edged[1687]: microsoft Jun 18 15:53:23 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:23Z [INFO] - Successfully queried system info Jun 18 15:53:23 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:23Z [INFO] - [mgmt] - - - [2021-06-18 15:53:23.688955 UTC] "GET /systeminfo?api-version=2020-07-07 HTTP/1.1" 200 OK 306 "-" "-" auth_id(-) Jun 18 15:53:24 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:24Z [INFO] - Querying system info... Jun 18 15:53:24 di-10-ubuntu1804 aziot-edged[1687]: microsoft Jun 18 15:53:24 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:24Z [INFO] - Successfully queried system info Jun 18 15:53:24 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:24Z [INFO] - [mgmt] - - - [2021-06-18 15:53:24.402640900 UTC] "GET /systeminfo?api-version=2020-07-07 HTTP/1.1" 200 OK 306 "-" "-" auth_id(-) Jun 18 15:53:24 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:24Z [INFO] - <-- POST /key?api-version=2020-09-01 {"content-length": "74", "content-type": "application/json"} Jun 18 15:53:24 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:24Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:24 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:24Z [INFO] - <-- POST /decrypt?api-version=2020-09-01 {"content-length": "1831", "content-type": "application/json"} Jun 18 15:53:24 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:24Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:24 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:24Z [INFO] - [work] - - - [2021-06-18 15:53:24.478795300 UTC] "POST /modules/%24edgeAgent/genid/637596214573249542/decrypt?api-version=2020-07-07 HTTP/1.1" 200 OK 1404 "-" "-" auth_id(-) Jun 18 15:53:24 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:24Z [INFO] - [mgmt] - - - [2021-06-18 15:53:24.609995 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:53:29 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:29Z [INFO] - <-- POST /key?api-version=2020-09-01 {"content-length": "74", "content-type": "application/json"} Jun 18 15:53:29 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:29Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:29 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:29Z [INFO] - <-- POST /decrypt?api-version=2020-09-01 {"content-length": "1831", "content-type": "application/json"} Jun 18 15:53:29 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:29Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:29 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:29Z [INFO] - [work] - - - [2021-06-18 15:53:29.635668500 UTC] "POST /modules/%24edgeAgent/genid/637596214573249542/decrypt?api-version=2020-07-07 HTTP/1.1" 200 OK 1404 "-" "-" auth_id(-) Jun 18 15:53:29 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:29Z [INFO] - Stopping module SimulatedTemperatureSensor... Jun 18 15:53:29 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:29Z [WARN] - Could not stop module SimulatedTemperatureSensor Jun 18 15:53:29 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:29Z [WARN] - caused by: Target of operation already in this state Jun 18 15:53:29 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:29Z [INFO] - [mgmt] - - - [2021-06-18 15:53:29.783776 UTC] "POST /modules/SimulatedTemperatureSensor/stop?api-version=2020-07-07 HTTP/1.1" 304 Not Modified 0 "-" "-" auth_id(-) Jun 18 15:53:29 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:29Z [INFO] - Starting module SimulatedTemperatureSensor... Jun 18 15:53:30 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:30Z [INFO] - Successfully started module SimulatedTemperatureSensor Jun 18 15:53:30 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:30Z [INFO] - [mgmt] - - - [2021-06-18 15:53:30.251174300 UTC] "POST /modules/SimulatedTemperatureSensor/start?api-version=2020-07-07 HTTP/1.1" 204 No Content - "-" "-" auth_id(-) Jun 18 15:53:30 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:30Z [INFO] - Stopping module edgeHub... Jun 18 15:53:30 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:30Z [WARN] - Could not stop module edgeHub Jun 18 15:53:30 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:30Z [WARN] - caused by: Target of operation already in this state Jun 18 15:53:30 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:30Z [INFO] - [mgmt] - - - [2021-06-18 15:53:30.276250700 UTC] "POST /modules/edgeHub/stop?api-version=2020-07-07 HTTP/1.1" 304 Not Modified 0 "-" "-" auth_id(-) Jun 18 15:53:30 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:30Z [INFO] - Starting module edgeHub... Jun 18 15:53:30 di-10-ubuntu1804 aziot-certd[1768]: 2021-06-18T15:53:30Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"} Jun 18 15:53:30 di-10-ubuntu1804 aziot-certd[1768]: 2021-06-18T15:53:30Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:30 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:30Z [INFO] - [work] - - - [2021-06-18 15:53:30.824054100 UTC] "GET /trust-bundle?api-version=2018-06-28 HTTP/1.1" 200 OK 8024 "-" "-" auth_id(-) Jun 18 15:53:31 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:31Z [INFO] - Successfully started module edgeHub Jun 18 15:53:31 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:31Z [INFO] - [mgmt] - - - [2021-06-18 15:53:31.095404800 UTC] "POST /modules/edgeHub/start?api-version=2020-07-07 HTTP/1.1" 204 No Content - "-" "-" auth_id(-) Jun 18 15:53:31 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:31Z [INFO] - <-- POST /key?api-version=2020-09-01 {"content-length": "74", "content-type": "application/json"} Jun 18 15:53:31 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:31Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:31 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:31Z [INFO] - <-- POST /encrypt?api-version=2020-09-01 {"content-length": "1806", "content-type": "application/json"} Jun 18 15:53:31 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:31Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:31 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:31Z [INFO] - [work] - - - [2021-06-18 15:53:31.138996 UTC] "POST /modules/%24edgeAgent/genid/637596214573249542/encrypt?api-version=2020-07-07 HTTP/1.1" 200 OK 1429 "-" "-" auth_id(-) Jun 18 15:53:31 di-10-ubuntu1804 aziot-certd[1768]: 2021-06-18T15:53:31Z [INFO] - <-- GET /certificates/aziot-edged-ca?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"} Jun 18 15:53:31 di-10-ubuntu1804 aziot-certd[1768]: 2021-06-18T15:53:31Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:31 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:31Z [INFO] - <-- GET /keypair/aziot-edged-ca?api-version=2020-09-01 {} Jun 18 15:53:31 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:31Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:31 di-10-ubuntu1804 aziot-certd[1768]: 2021-06-18T15:53:31Z [INFO] - <-- POST /certificates?api-version=2020-09-01 {"content-type": "application/json", "host": "2f72756e2f617a696f742f63657274642e736f636b:0", "content-length": "1392"} Jun 18 15:53:31 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:31Z [INFO] - <-- POST /parameters/algorithm?api-version=2020-09-01 {"content-length": "252", "content-type": "application/json"} Jun 18 15:53:31 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:31Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:31 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:31Z [INFO] - <-- POST /parameters/rsa-modulus?api-version=2020-09-01 {"content-length": "252", "content-type": "application/json"} Jun 18 15:53:31 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:31Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:31 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:31Z [INFO] - <-- POST /parameters/rsa-exponent?api-version=2020-09-01 {"content-length": "252", "content-type": "application/json"} Jun 18 15:53:31 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:31Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:31 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:31Z [INFO] - <-- POST /encrypt?api-version=2020-09-01 {"content-length": "359", "content-type": "application/json"} Jun 18 15:53:31 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:31Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:31 di-10-ubuntu1804 aziot-certd[1768]: 2021-06-18T15:53:31Z [INFO] - --> 201 {"content-type": "application/json"} Jun 18 15:53:31 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:31Z [INFO] - [work] - - - [2021-06-18 15:53:31.615286400 UTC] "POST /modules/%24edgeHub/genid/637596214573249542/certificate/server?api-version=2019-01-30 HTTP/1.1" 201 Created 9272 "-" "-" auth_id(-) Jun 18 15:53:31 di-10-ubuntu1804 aziot-certd[1768]: 2021-06-18T15:53:31Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"} Jun 18 15:53:31 di-10-ubuntu1804 aziot-certd[1768]: 2021-06-18T15:53:31Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:31 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:31Z [INFO] - [work] - - - [2021-06-18 15:53:31.865186900 UTC] "GET /trust-bundle?api-version=2019-01-30 HTTP/1.1" 200 OK 8024 "-" "-" auth_id(-) Jun 18 15:53:32 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:32Z [INFO] - <-- POST /key?api-version=2020-09-01 {"content-length": "74", "content-type": "application/json"} Jun 18 15:53:32 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:32Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:32 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:32Z [INFO] - <-- POST /decrypt?api-version=2020-09-01 {"content-length": "1071", "content-type": "application/json"} Jun 18 15:53:32 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:32Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:32 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:32Z [INFO] - [work] - - - [2021-06-18 15:53:32.704957300 UTC] "POST /modules/%24edgeHub/genid/637596214573249542/decrypt?api-version=2019-01-30 HTTP/1.1" 200 OK 652 "-" "-" auth_id(-) Jun 18 15:53:32 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:32Z [INFO] - <-- POST /key?api-version=2020-09-01 {"content-length": "74", "content-type": "application/json"} Jun 18 15:53:32 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:32Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:32 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:32Z [INFO] - <-- POST /decrypt?api-version=2020-09-01 {"content-length": "1031", "content-type": "application/json"} Jun 18 15:53:32 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:32Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:32 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:32Z [INFO] - [work] - - - [2021-06-18 15:53:32.776477700 UTC] "POST /modules/%24edgeHub/genid/637596214573249542/decrypt?api-version=2019-01-30 HTTP/1.1" 200 OK 608 "-" "-" auth_id(-) Jun 18 15:53:32 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:32Z [INFO] - <-- POST /key?api-version=2020-09-01 {"content-length": "74", "content-type": "application/json"} Jun 18 15:53:32 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:32Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:32 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:32Z [INFO] - <-- POST /decrypt?api-version=2020-09-01 {"content-length": "1023", "content-type": "application/json"} Jun 18 15:53:32 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:32Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:32 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:32Z [INFO] - [work] - - - [2021-06-18 15:53:32.793058800 UTC] "POST /modules/%24edgeHub/genid/637596214573249542/decrypt?api-version=2019-01-30 HTTP/1.1" 200 OK 604 "-" "-" auth_id(-) Jun 18 15:53:32 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:32Z [INFO] - <-- POST /key?api-version=2020-09-01 {"content-length": "74", "content-type": "application/json"} Jun 18 15:53:32 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:32Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:32 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:32Z [INFO] - <-- POST /decrypt?api-version=2020-09-01 {"content-length": "1071", "content-type": "application/json"} Jun 18 15:53:32 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:32Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:32 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:32Z [INFO] - [work] - - - [2021-06-18 15:53:32.809887400 UTC] "POST /modules/%24edgeHub/genid/637596214573249542/decrypt?api-version=2019-01-30 HTTP/1.1" 200 OK 652 "-" "-" auth_id(-) Jun 18 15:53:32 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:32Z [INFO] - <-- POST /key?api-version=2020-09-01 {"content-length": "74", "content-type": "application/json"} Jun 18 15:53:32 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:32Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:32 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:32Z [INFO] - <-- POST /decrypt?api-version=2020-09-01 {"content-length": "1143", "content-type": "application/json"} Jun 18 15:53:32 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:32Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:32 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:32Z [INFO] - [work] - - - [2021-06-18 15:53:32.825770900 UTC] "POST /modules/%24edgeHub/genid/637596214573249542/decrypt?api-version=2019-01-30 HTTP/1.1" 200 OK 724 "-" "-" auth_id(-) Jun 18 15:53:32 di-10-ubuntu1804 aziot-identityd[1716]: 2021-06-18T15:53:32Z [INFO] - <-- GET /identities/modules/$edgeHub?api-version=2020-09-01&type=aziot {"content-type": "application/json", "host": "2f72756e2f617a696f742f6964656e74697479642e736f636b:0", "content-length": "38"} Jun 18 15:53:32 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:32Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"} Jun 18 15:53:32 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:32Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:32 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:32Z [INFO] - <-- POST /sign?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "370"} Jun 18 15:53:32 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:32Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:32 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:32Z [INFO] - <-- POST /key?api-version=2020-09-01 {"content-length": "74", "content-type": "application/json"} Jun 18 15:53:32 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:32Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:32 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:32Z [INFO] - <-- POST /decrypt?api-version=2020-09-01 {"content-length": "1455", "content-type": "application/json"} Jun 18 15:53:32 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:32Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:32 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:32Z [INFO] - [work] - - - [2021-06-18 15:53:32.991491200 UTC] "POST /modules/%24edgeHub/genid/637596214573249542/decrypt?api-version=2019-01-30 HTTP/1.1" 200 OK 1036 "-" "-" auth_id(-) Jun 18 15:53:33 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:33Z [INFO] - <-- POST /key?api-version=2020-09-01 {"content-length": "74", "content-type": "application/json"} Jun 18 15:53:33 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:33Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:33 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:33Z [INFO] - <-- POST /encrypt?api-version=2020-09-01 {"content-length": "1190", "content-type": "application/json"} Jun 18 15:53:33 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:33Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:33 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:33Z [INFO] - [work] - - - [2021-06-18 15:53:33.053095900 UTC] "POST /modules/%24edgeHub/genid/637596214573249542/encrypt?api-version=2019-01-30 HTTP/1.1" 200 OK 817 "-" "-" auth_id(-) Jun 18 15:53:33 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:33Z [INFO] - <-- POST /key?api-version=2020-09-01 {"content-length": "74", "content-type": "application/json"} Jun 18 15:53:33 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:33Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:33 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:33Z [INFO] - <-- POST /decrypt?api-version=2020-09-01 {"content-length": "1215", "content-type": "application/json"} Jun 18 15:53:33 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:33Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:33 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:33Z [INFO] - [work] - - - [2021-06-18 15:53:33.083191800 UTC] "POST /modules/%24edgeHub/genid/637596214573249542/decrypt?api-version=2019-01-30 HTTP/1.1" 200 OK 792 "-" "-" auth_id(-) Jun 18 15:53:33 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:33Z [INFO] - <-- POST /key?api-version=2020-09-01 {"content-length": "74", "content-type": "application/json"} Jun 18 15:53:33 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:33Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:33 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:33Z [INFO] - <-- POST /encrypt?api-version=2020-09-01 {"content-length": "1370", "content-type": "application/json"} Jun 18 15:53:33 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:33Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:33 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:33Z [INFO] - [work] - - - [2021-06-18 15:53:33.099316300 UTC] "POST /modules/%24edgeHub/genid/637596214573249542/encrypt?api-version=2019-01-30 HTTP/1.1" 200 OK 997 "-" "-" auth_id(-) Jun 18 15:53:33 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:33Z [INFO] - <-- POST /key?api-version=2020-09-01 {"content-length": "74", "content-type": "application/json"} Jun 18 15:53:33 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:33Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:33 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:33Z [INFO] - <-- POST /decrypt?api-version=2020-09-01 {"content-length": "1395", "content-type": "application/json"} Jun 18 15:53:33 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:33Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:33 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:33Z [INFO] - [work] - - - [2021-06-18 15:53:33.178534800 UTC] "POST /modules/%24edgeHub/genid/637596214573249542/decrypt?api-version=2019-01-30 HTTP/1.1" 200 OK 972 "-" "-" auth_id(-) Jun 18 15:53:34 di-10-ubuntu1804 aziot-identityd[1716]: 2021-06-18T15:53:34Z [INFO] - <-- GET /identities/modules/SimulatedTemperatureSensor?api-version=2020-09-01&type=aziot {"content-type": "application/json", "host": "2f72756e2f617a696f742f6964656e74697479642e736f636b:0", "content-length": "56"} Jun 18 15:53:36 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:36Z [INFO] - [mgmt] - - - [2021-06-18 15:53:36.154889600 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:53:37 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:37Z [INFO] - <-- POST /key?api-version=2020-09-01 {"content-length": "74", "content-type": "application/json"} Jun 18 15:53:37 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:37Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:37 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:37Z [INFO] - <-- POST /decrypt?api-version=2020-09-01 {"content-length": "1395", "content-type": "application/json"} Jun 18 15:53:37 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:37Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:37 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:37Z [INFO] - [work] - - - [2021-06-18 15:53:37.867708700 UTC] "POST /modules/%24edgeHub/genid/637596214573249542/decrypt?api-version=2019-01-30 HTTP/1.1" 200 OK 972 "-" "-" auth_id(-) Jun 18 15:53:37 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:37Z [INFO] - <-- POST /key?api-version=2020-09-01 {"content-length": "74", "content-type": "application/json"} Jun 18 15:53:37 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:37Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:37 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:37Z [INFO] - <-- POST /decrypt?api-version=2020-09-01 {"content-length": "1395", "content-type": "application/json"} Jun 18 15:53:37 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:37Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:37 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:37Z [INFO] - [work] - - - [2021-06-18 15:53:37.886084 UTC] "POST /modules/%24edgeHub/genid/637596214573249542/decrypt?api-version=2019-01-30 HTTP/1.1" 200 OK 972 "-" "-" auth_id(-) Jun 18 15:53:46 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:46Z [INFO] - [mgmt] - - - [2021-06-18 15:53:46.166082700 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:53:53 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:53Z [INFO] - <-- POST /key?api-version=2020-09-01 {"content-length": "74", "content-type": "application/json"} Jun 18 15:53:53 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:53Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:53 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:53Z [INFO] - <-- POST /decrypt?api-version=2020-09-01 {"content-length": "1395", "content-type": "application/json"} Jun 18 15:53:53 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:53Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:53 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:53Z [INFO] - [work] - - - [2021-06-18 15:53:53.884290300 UTC] "POST /modules/%24edgeHub/genid/637596214573249542/decrypt?api-version=2019-01-30 HTTP/1.1" 200 OK 972 "-" "-" auth_id(-) Jun 18 15:53:53 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:53Z [INFO] - <-- POST /key?api-version=2020-09-01 {"content-length": "74", "content-type": "application/json"} Jun 18 15:53:53 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:53Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:53 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:53Z [INFO] - <-- POST /decrypt?api-version=2020-09-01 {"content-length": "1395", "content-type": "application/json"} Jun 18 15:53:53 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:53Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:53 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:53Z [INFO] - [work] - - - [2021-06-18 15:53:53.932463200 UTC] "POST /modules/%24edgeHub/genid/637596214573249542/decrypt?api-version=2019-01-30 HTTP/1.1" 200 OK 972 "-" "-" auth_id(-) Jun 18 15:53:53 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:53Z [INFO] - <-- POST /key?api-version=2020-09-01 {"content-length": "74", "content-type": "application/json"} Jun 18 15:53:53 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:53Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:53 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:53Z [INFO] - <-- POST /encrypt?api-version=2020-09-01 {"content-length": "1390", "content-type": "application/json"} Jun 18 15:53:53 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:53Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:53 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:53Z [INFO] - [work] - - - [2021-06-18 15:53:53.949320900 UTC] "POST /modules/%24edgeHub/genid/637596214573249542/encrypt?api-version=2019-01-30 HTTP/1.1" 200 OK 1013 "-" "-" auth_id(-) Jun 18 15:53:53 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:53Z [INFO] - <-- POST /key?api-version=2020-09-01 {"content-length": "74", "content-type": "application/json"} Jun 18 15:53:53 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:53Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:53 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:53Z [INFO] - <-- POST /decrypt?api-version=2020-09-01 {"content-length": "1411", "content-type": "application/json"} Jun 18 15:53:53 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:53Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:53 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:53Z [INFO] - [work] - - - [2021-06-18 15:53:53.967525600 UTC] "POST /modules/%24edgeHub/genid/637596214573249542/decrypt?api-version=2019-01-30 HTTP/1.1" 200 OK 992 "-" "-" auth_id(-) Jun 18 15:53:53 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:53Z [INFO] - <-- POST /key?api-version=2020-09-01 {"content-length": "74", "content-type": "application/json"} Jun 18 15:53:53 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:53Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:53 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:53Z [INFO] - <-- POST /encrypt?api-version=2020-09-01 {"content-length": "1482", "content-type": "application/json"} Jun 18 15:53:53 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:53Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:53 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:53Z [INFO] - [work] - - - [2021-06-18 15:53:53.983283300 UTC] "POST /modules/%24edgeHub/genid/637596214573249542/encrypt?api-version=2019-01-30 HTTP/1.1" 200 OK 1109 "-" "-" auth_id(-) Jun 18 15:53:56 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:56Z [INFO] - [mgmt] - - - [2021-06-18 15:53:56.173868800 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:53:58 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:58Z [INFO] - <-- POST /key?api-version=2020-09-01 {"content-length": "74", "content-type": "application/json"} Jun 18 15:53:58 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:58Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:58 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:58Z [INFO] - <-- POST /decrypt?api-version=2020-09-01 {"content-length": "1507", "content-type": "application/json"} Jun 18 15:53:58 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:58Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:58 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:58Z [INFO] - [work] - - - [2021-06-18 15:53:58.898650900 UTC] "POST /modules/%24edgeHub/genid/637596214573249542/decrypt?api-version=2019-01-30 HTTP/1.1" 200 OK 1084 "-" "-" auth_id(-) Jun 18 15:53:58 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:58Z [INFO] - <-- POST /key?api-version=2020-09-01 {"content-length": "74", "content-type": "application/json"} Jun 18 15:53:58 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:58Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:58 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:58Z [INFO] - <-- POST /decrypt?api-version=2020-09-01 {"content-length": "1507", "content-type": "application/json"} Jun 18 15:53:58 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:53:58Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:53:58 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:53:58Z [INFO] - [work] - - - [2021-06-18 15:53:58.932220600 UTC] "POST /modules/%24edgeHub/genid/637596214573249542/decrypt?api-version=2019-01-30 HTTP/1.1" 200 OK 1084 "-" "-" auth_id(-) Jun 18 15:54:06 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:54:06Z [INFO] - [mgmt] - - - [2021-06-18 15:54:06.181070200 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:54:16 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:54:16Z [INFO] - [mgmt] - - - [2021-06-18 15:54:16.188670800 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:54:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:54:21Z [INFO] - Checking edge runtime status Jun 18 15:54:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:54:21Z [INFO] - Edge runtime is running. Jun 18 15:54:23 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:54:23Z [INFO] - Querying system resources... Jun 18 15:54:26 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:54:26Z [INFO] - [mgmt] - - - [2021-06-18 15:54:26.212409900 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:54:28 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:54:28Z [INFO] - [mgmt] - - - [2021-06-18 15:54:28.715399900 UTC] "GET /systeminfo/resources?api-version=2020-07-07 HTTP/1.1" 200 OK 8934 "-" "-" auth_id(-) Jun 18 15:54:36 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:54:36Z [INFO] - [mgmt] - - - [2021-06-18 15:54:36.221346300 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:54:46 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:54:46Z [INFO] - [mgmt] - - - [2021-06-18 15:54:46.221602200 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:54:56 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:54:56Z [INFO] - [mgmt] - - - [2021-06-18 15:54:56.220666200 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:55:06 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:55:06Z [INFO] - [mgmt] - - - [2021-06-18 15:55:06.225832500 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:55:15 di-10-ubuntu1804 aziot-identityd[1716]: 2021-06-18T15:55:15Z [INFO] - <-- GET /identities/modules/SimulatedTemperatureSensor?api-version=2020-09-01&type=aziot {"content-type": "application/json", "host": "2f72756e2f617a696f742f6964656e74697479642e736f636b:0", "content-length": "56"} Jun 18 15:55:16 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:55:16Z [INFO] - [mgmt] - - - [2021-06-18 15:55:16.232729800 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:55:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:55:21Z [INFO] - Checking edge runtime status Jun 18 15:55:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:55:21Z [INFO] - Edge runtime is running. Jun 18 15:55:23 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:55:23Z [INFO] - [mgmt] - - - [2021-06-18 15:55:23.785133800 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "iotedge/0.1.0" auth_id(-) Jun 18 15:55:26 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:55:26Z [INFO] - [mgmt] - - - [2021-06-18 15:55:26.231773500 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:55:36 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:55:36Z [INFO] - [mgmt] - - - [2021-06-18 15:55:36.237280800 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:55:40 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:55:40Z [INFO] - Getting logs for module SimulatedTemperatureSensor... Jun 18 15:55:40 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:55:40Z [INFO] - Successfully got logs for module SimulatedTemperatureSensor Jun 18 15:55:40 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:55:40Z [INFO] - [mgmt] - - - [2021-06-18 15:55:40.174371700 UTC] "GET /modules/SimulatedTemperatureSensor/logs?api-version=2020-07-07&follow=true&tail=all×tamps=false&since=1623945340 HTTP/1.1" 200 OK - "-" "iotedge/0.1.0" auth_id(-) Jun 18 15:55:43 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:55:43Z [INFO] - <-- GET /key/aziot_identityd_master_id?api-version=2020-09-01 {"host": "keyd.sock"} Jun 18 15:55:43 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:55:43Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:55:43 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:55:43Z [INFO] - <-- POST /derivedkey?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "336"} Jun 18 15:55:43 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:55:43Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:55:43 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:55:43Z [INFO] - <-- POST /derivedkey/export?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "724"} Jun 18 15:55:43 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:55:43Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:55:43 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:55:43Z [INFO] - <-- POST /derivedkey?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "340"} Jun 18 15:55:43 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:55:43Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:55:43 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:55:43Z [INFO] - <-- POST /derivedkey/export?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "736"} Jun 18 15:55:43 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:55:43Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:55:43 di-10-ubuntu1804 aziot-identityd[1716]: 2021-06-18T15:55:43Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:55:43 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:55:43Z [INFO] - <-- POST /sign?api-version=2020-09-01 {"content-length": "894", "content-type": "application/json"} Jun 18 15:55:43 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:55:43Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:55:43 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:55:43Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"} Jun 18 15:55:43 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:55:43Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:55:43 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:55:43Z [INFO] - [work] - - - [2021-06-18 15:55:43.800829900 UTC] "POST /modules/%24edgeHub/genid/637596214573249542/sign?api-version=2019-01-30 HTTP/1.1" 200 OK 57 "-" "-" auth_id(-) Jun 18 15:55:43 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:55:43Z [INFO] - <-- POST /sign?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "370"} Jun 18 15:55:43 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:55:43Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:55:46 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:55:46Z [INFO] - [mgmt] - - - [2021-06-18 15:55:46.240243400 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:55:56 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:55:56Z [INFO] - [mgmt] - - - [2021-06-18 15:55:56.243213800 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:56:06 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:56:06Z [INFO] - [mgmt] - - - [2021-06-18 15:56:06.246597100 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:56:16 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:56:16Z [INFO] - [mgmt] - - - [2021-06-18 15:56:16.251484200 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:56:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:56:21Z [INFO] - Checking edge runtime status Jun 18 15:56:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:56:21Z [INFO] - Edge runtime is running. Jun 18 15:56:26 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:56:26Z [INFO] - [mgmt] - - - [2021-06-18 15:56:26.252485400 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:56:36 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:56:36Z [INFO] - [mgmt] - - - [2021-06-18 15:56:36.255442600 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:56:46 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:56:46Z [INFO] - [mgmt] - - - [2021-06-18 15:56:46.257836800 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:56:56 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:56:56Z [INFO] - [mgmt] - - - [2021-06-18 15:56:56.262248 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:56:56 di-10-ubuntu1804 aziot-identityd[1716]: 2021-06-18T15:56:56Z [INFO] - <-- GET /identities/modules/SimulatedTemperatureSensor?api-version=2020-09-01&type=aziot {"content-type": "application/json", "host": "2f72756e2f617a696f742f6964656e74697479642e736f636b:0", "content-length": "56"} Jun 18 15:57:06 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:57:06Z [INFO] - [mgmt] - - - [2021-06-18 15:57:06.261076700 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:57:16 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:57:16Z [INFO] - [mgmt] - - - [2021-06-18 15:57:16.267837700 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:57:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:57:21Z [INFO] - Checking edge runtime status Jun 18 15:57:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:57:21Z [INFO] - Edge runtime is running. Jun 18 15:57:26 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:57:26Z [INFO] - [mgmt] - - - [2021-06-18 15:57:26.265506300 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:57:36 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:57:36Z [INFO] - [mgmt] - - - [2021-06-18 15:57:36.269634400 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:57:46 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:57:46Z [INFO] - [mgmt] - - - [2021-06-18 15:57:46.275759900 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:57:54 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:57:54Z [INFO] - <-- GET /key/aziot_identityd_master_id?api-version=2020-09-01 {"host": "keyd.sock"} Jun 18 15:57:54 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:57:54Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:57:54 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:57:54Z [INFO] - <-- POST /derivedkey?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "360"} Jun 18 15:57:54 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:57:54Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:57:54 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:57:54Z [INFO] - <-- POST /derivedkey/export?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "820"} Jun 18 15:57:54 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:57:54Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:57:54 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:57:54Z [INFO] - <-- POST /derivedkey?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "364"} Jun 18 15:57:54 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:57:54Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:57:54 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:57:54Z [INFO] - <-- POST /derivedkey/export?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "828"} Jun 18 15:57:54 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:57:54Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:57:54 di-10-ubuntu1804 aziot-identityd[1716]: 2021-06-18T15:57:54Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:57:54 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:57:54Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"} Jun 18 15:57:54 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:57:54Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:57:54 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:57:54Z [INFO] - <-- POST /sign?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "370"} Jun 18 15:57:54 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:57:54Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:57:54 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:57:54Z [INFO] - <-- POST /sign?api-version=2020-09-01 {"content-length": "1006", "content-type": "application/json"} Jun 18 15:57:54 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T15:57:54Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:57:54 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:57:54Z [INFO] - [work] - - - [2021-06-18 15:57:54.871792600 UTC] "POST /modules/SimulatedTemperatureSensor/genid/637596270173859214/sign?api-version=2018-06-28 HTTP/1.1" 200 OK 57 "-" "-" auth_id(-) Jun 18 15:57:56 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:57:56Z [INFO] - [mgmt] - - - [2021-06-18 15:57:56.272380900 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:58:06 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:58:06Z [INFO] - [mgmt] - - - [2021-06-18 15:58:06.280340200 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:58:16 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:58:16Z [INFO] - [mgmt] - - - [2021-06-18 15:58:16.284383200 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:58:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:58:21Z [INFO] - Checking edge runtime status Jun 18 15:58:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:58:21Z [INFO] - Edge runtime is running. Jun 18 15:58:26 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:58:26Z [INFO] - [mgmt] - - - [2021-06-18 15:58:26.285301500 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:58:33 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:58:33Z [INFO] - Getting logs for module SimulatedTemperatureSensor... Jun 18 15:58:33 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:58:33Z [INFO] - Successfully got logs for module SimulatedTemperatureSensor Jun 18 15:58:33 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:58:33Z [INFO] - [mgmt] - - - [2021-06-18 15:58:33.814780600 UTC] "GET /modules/SimulatedTemperatureSensor/logs?api-version=2020-07-07&follow=true&tail=all×tamps=false&since=1623945513 HTTP/1.1" 200 OK - "-" "iotedge/0.1.0" auth_id(-) Jun 18 15:58:36 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:58:36Z [INFO] - [mgmt] - - - [2021-06-18 15:58:36.290134900 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:58:36 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:58:36Z [ERR!] - server connection error: (unknown) Jun 18 15:58:36 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:58:36Z [ERR!] - error writing a body to connection: Broken pipe (os error 32) Jun 18 15:58:36 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:58:36Z [ERR!] - server connection error: (unknown) Jun 18 15:58:36 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:58:36Z [ERR!] - error writing a body to connection: Broken pipe (os error 32) Jun 18 15:58:36 di-10-ubuntu1804 aziot-certd[1768]: 2021-06-18T15:58:36Z [INFO] - <-- GET /certificates/aziot-edged-trust-bundle?api-version=2020-09-01 {"host": "2f72756e2f617a696f742f63657274642e736f636b:0"} Jun 18 15:58:36 di-10-ubuntu1804 aziot-certd[1768]: 2021-06-18T15:58:36Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 15:58:36 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:58:36Z [INFO] - [work] - - - [2021-06-18 15:58:36.980713600 UTC] "GET /trust-bundle?api-version=2018-06-28 HTTP/1.1" 200 OK 8024 "-" "-" auth_id(-) Jun 18 15:58:37 di-10-ubuntu1804 aziot-identityd[1716]: 2021-06-18T15:58:37Z [INFO] - <-- GET /identities/modules/SimulatedTemperatureSensor?api-version=2020-09-01&type=aziot {"content-type": "application/json", "host": "2f72756e2f617a696f742f6964656e74697479642e736f636b:0", "content-length": "56"} Jun 18 15:58:45 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:58:45Z [INFO] - Getting logs for module edgeHub... Jun 18 15:58:45 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:58:45Z [INFO] - Successfully got logs for module edgeHub Jun 18 15:58:45 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:58:45Z [INFO] - [mgmt] - - - [2021-06-18 15:58:45.433215200 UTC] "GET /modules/edgeHub/logs?api-version=2020-07-07&follow=true&tail=all×tamps=false&since=1623945525 HTTP/1.1" 200 OK - "-" "iotedge/0.1.0" auth_id(-) Jun 18 15:58:46 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:58:46Z [INFO] - [mgmt] - - - [2021-06-18 15:58:46.293646200 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:58:56 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:58:56Z [INFO] - [mgmt] - - - [2021-06-18 15:58:56.296314300 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:59:06 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:59:06Z [INFO] - [mgmt] - - - [2021-06-18 15:59:06.299382600 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:59:16 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:59:16Z [INFO] - [mgmt] - - - [2021-06-18 15:59:16.299946300 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:59:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:59:21Z [INFO] - Checking edge runtime status Jun 18 15:59:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:59:21Z [INFO] - Edge runtime is running. Jun 18 15:59:26 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:59:26Z [INFO] - [mgmt] - - - [2021-06-18 15:59:26.298531100 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:59:28 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:59:28Z [INFO] - Querying system resources... Jun 18 15:59:33 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:59:33Z [INFO] - [mgmt] - - - [2021-06-18 15:59:33.777958900 UTC] "GET /systeminfo/resources?api-version=2020-07-07 HTTP/1.1" 200 OK 8972 "-" "-" auth_id(-) Jun 18 15:59:36 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:59:36Z [INFO] - [mgmt] - - - [2021-06-18 15:59:36.298376300 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:59:46 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:59:46Z [INFO] - [mgmt] - - - [2021-06-18 15:59:46.299891100 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 15:59:56 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T15:59:56Z [INFO] - [mgmt] - - - [2021-06-18 15:59:56.302520500 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 16:00:05 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T16:00:05Z [INFO] - <-- GET /key/aziot_identityd_master_id?api-version=2020-09-01 {"host": "keyd.sock"} Jun 18 16:00:05 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T16:00:05Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 16:00:05 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T16:00:05Z [INFO] - <-- POST /derivedkey?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "360"} Jun 18 16:00:05 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T16:00:05Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 16:00:05 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T16:00:05Z [INFO] - <-- POST /derivedkey/export?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "820"} Jun 18 16:00:05 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T16:00:05Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 16:00:05 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T16:00:05Z [INFO] - <-- POST /derivedkey?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "364"} Jun 18 16:00:05 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T16:00:05Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 16:00:05 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T16:00:05Z [INFO] - <-- POST /derivedkey/export?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "828"} Jun 18 16:00:05 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T16:00:05Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 16:00:05 di-10-ubuntu1804 aziot-identityd[1716]: 2021-06-18T16:00:05Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 16:00:05 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T16:00:05Z [INFO] - <-- GET /key/device-id?api-version=2020-09-01 {"host": "keyd.sock"} Jun 18 16:00:05 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T16:00:05Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 16:00:05 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T16:00:05Z [INFO] - <-- POST /sign?api-version=2020-09-01 {"content-type": "application/json", "host": "keyd.sock", "content-length": "370"} Jun 18 16:00:05 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T16:00:05Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 16:00:05 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T16:00:05Z [INFO] - <-- POST /sign?api-version=2020-09-01 {"content-length": "1006", "content-type": "application/json"} Jun 18 16:00:05 di-10-ubuntu1804 aziot-keyd[1753]: 2021-06-18T16:00:05Z [INFO] - --> 200 {"content-type": "application/json"} Jun 18 16:00:05 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T16:00:05Z [INFO] - [work] - - - [2021-06-18 16:00:05.942352 UTC] "POST /modules/SimulatedTemperatureSensor/genid/637596270173859214/sign?api-version=2018-06-28 HTTP/1.1" 200 OK 57 "-" "-" auth_id(-) Jun 18 16:00:06 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T16:00:06Z [INFO] - [mgmt] - - - [2021-06-18 16:00:06.304390 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 16:00:16 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T16:00:16Z [INFO] - [mgmt] - - - [2021-06-18 16:00:16.306484800 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 16:00:17 di-10-ubuntu1804 aziot-identityd[1716]: 2021-06-18T16:00:17Z [INFO] - <-- GET /identities/modules/SimulatedTemperatureSensor?api-version=2020-09-01&type=aziot {"content-type": "application/json", "host": "2f72756e2f617a696f742f6964656e74697479642e736f636b:0", "content-length": "56"} Jun 18 16:00:19 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T16:00:19Z [INFO] - Getting logs for module edgeAgent... Jun 18 16:00:19 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T16:00:19Z [INFO] - Successfully got logs for module edgeAgent Jun 18 16:00:19 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T16:00:19Z [INFO] - [mgmt] - - - [2021-06-18 16:00:19.317348600 UTC] "GET /modules/edgeAgent/logs?api-version=2020-07-07&follow=true&tail=all×tamps=false&since=1623945619 HTTP/1.1" 200 OK - "-" "iotedge/0.1.0" auth_id(-) Jun 18 16:00:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T16:00:21Z [INFO] - Checking edge runtime status Jun 18 16:00:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T16:00:21Z [INFO] - Edge runtime is running. Jun 18 16:00:26 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T16:00:26Z [INFO] - [mgmt] - - - [2021-06-18 16:00:26.309201300 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 16:00:36 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T16:00:36Z [INFO] - [mgmt] - - - [2021-06-18 16:00:36.313670700 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 16:00:46 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T16:00:46Z [INFO] - [mgmt] - - - [2021-06-18 16:00:46.319092300 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 16:00:56 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T16:00:56Z [INFO] - [mgmt] - - - [2021-06-18 16:00:56.323151800 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 16:01:06 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T16:01:06Z [INFO] - [mgmt] - - - [2021-06-18 16:01:06.323223700 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 16:01:16 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T16:01:16Z [INFO] - [mgmt] - - - [2021-06-18 16:01:16.327068200 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 16:01:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T16:01:21Z [INFO] - Checking edge runtime status Jun 18 16:01:21 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T16:01:21Z [INFO] - Edge runtime is running. Jun 18 16:01:26 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T16:01:26Z [INFO] - [mgmt] - - - [2021-06-18 16:01:26.329718600 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 16:01:36 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T16:01:36Z [INFO] - [mgmt] - - - [2021-06-18 16:01:36.329530600 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) Jun 18 16:01:39 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T16:01:39Z [ERR!] - server connection error: (unknown) Jun 18 16:01:39 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T16:01:39Z [ERR!] - error writing a body to connection: Broken pipe (os error 32) Jun 18 16:01:46 di-10-ubuntu1804 aziot-edged[1687]: 2021-06-18T16:01:46Z [INFO] - [mgmt] - - - [2021-06-18 16:01:46.332466400 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 1949 "-" "-" auth_id(-) ```
edge-agent logs ``` 2021-06-18 15:53:22 Changing ownership of management socket: /var/run/iotedge/mgmt.sock 2021-06-18 15:53:22.895 +00:00 Edge Agent Main() <6> 2021-06-18 15:53:23.114 +00:00 [INF] - Initializing Edge Agent. <6> 2021-06-18 15:53:23.290 +00:00 [INF] - Version - 1.2.1.43175312 (f0e7486e3fd527515999b9449313e55b733516c9) <6> 2021-06-18 15:53:23.291 +00:00 [INF] - █████╗ ███████╗██╗ ██╗██████╗ ███████╗ ██╔══██╗╚══███╔╝██║ ██║██╔══██╗██╔════╝ ███████║ ███╔╝ ██║ ██║██████╔╝█████╗ ██╔══██║ ███╔╝ ██║ ██║██╔══██╗██╔══╝ ██║ ██║███████╗╚██████╔╝██║ ██║███████╗ ╚═╝ ╚═╝╚══════╝ ╚═════╝ ╚═╝ ╚═╝╚══════╝ ██╗ ██████╗ ████████╗ ███████╗██████╗ ██████╗ ███████╗ ██║██╔═══██╗╚══██╔══╝ ██╔════╝██╔══██╗██╔════╝ ██╔════╝ ██║██║ ██║ ██║ █████╗ ██║ ██║██║ ███╗█████╗ ██║██║ ██║ ██║ ██╔══╝ ██║ ██║██║ ██║██╔══╝ ██║╚██████╔╝ ██║ ███████╗██████╔╝╚██████╔╝███████╗ ╚═╝ ╚═════╝ ╚═╝ ╚══════╝╚═════╝ ╚═════╝ ╚══════╝ <6> 2021-06-18 15:53:23.327 +00:00 [INF] - Experimental features configuration: {"Enabled":false,"DisableCloudSubscriptions":false} <6> 2021-06-18 15:53:23.501 +00:00 [INF] - Installing certificates [CN=IoTEdgeGatewayCA.ca:07/18/2021 14:20:53],[CN=Azure_IoT_Hub_Intermediate_Cert_Test_Only:07/18/2021 14:20:32],[CN=Azure_IoT_Hub_CA_Cert_Test_Only:07/18/2021 14:20:32],[CN=Azure_IoT_Hub_CA_Cert_Test_Only:07/18/2021 14:20:32] to Root <6> 2021-06-18 15:53:23.639 +00:00 [INF] - Starting metrics listener on Host: *, Port: 9600, Suffix: metrics <6> 2021-06-18 15:53:23.658 +00:00 [INF] - Updating performance metrics every 05m:00s <6> 2021-06-18 15:53:23.662 +00:00 [INF] - Started operation Get system resources <6> 2021-06-18 15:53:23.664 +00:00 [INF] - Collecting metadata metrics <6> 2021-06-18 15:53:23.716 +00:00 [INF] - Set metadata metrics: 1.2.1.43175312 (f0e7486e3fd527515999b9449313e55b733516c9), {"Enabled":false,"DisableCloudSubscriptions":false}, {"OperatingSystemType":"linux","Architecture":"x86_64","Version":"1.2.1","Provisioning":{"Type":"ProvisioningType","DynamicReprovisioning":false,"AlwaysReprovisionOnStartup":false},"ServerVersion":"20.10.6+azure","KernelVersion":"5.4.0-1048-azure","OperatingSystem":"Ubuntu 18.04.5 LTS","NumCpus":2,"Virtualized":"yes"}, True <6> 2021-06-18 15:53:23.729 +00:00 [INF] - Started operation Checkpoint Availability <6> 2021-06-18 15:53:23.733 +00:00 [INF] - Started operation refresh twin config <6> 2021-06-18 15:53:23.749 +00:00 [INF] - Edge agent attempting to connect to IoT Hub via Amqp_Tcp_Only... <6> 2021-06-18 15:53:24.178 +00:00 [INF] - Created persistent store at /tmp/edgeAgent <6> 2021-06-18 15:53:24.232 +00:00 [INF] - Started operation Metrics Scrape <6> 2021-06-18 15:53:24.232 +00:00 [INF] - Started operation Metrics Upload Scraping frequency: 01:00:00 Upload Frequency: 1.00:00:00 <6> 2021-06-18 15:53:24.423 +00:00 [INF] - Registering request handler UploadModuleLogs <6> 2021-06-18 15:53:24.423 +00:00 [INF] - Registering request handler GetModuleLogs <6> 2021-06-18 15:53:24.423 +00:00 [INF] - Registering request handler UploadSupportBundle <6> 2021-06-18 15:53:24.423 +00:00 [INF] - Registering request handler RestartModule <4> 2021-06-18 15:53:29.613 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <6> 2021-06-18 15:53:29.638 +00:00 [INF] - Obtained edge agent config from backup config file - /tmp/edgeAgent/backup.json <6> 2021-06-18 15:53:29.767 +00:00 [INF] - Plan execution started for deployment 2 <6> 2021-06-18 15:53:29.769 +00:00 [INF] - Executing command: "Command Group: (\n [Stop module SimulatedTemperatureSensor]\n [Start module SimulatedTemperatureSensor]\n [Saving SimulatedTemperatureSensor to store]\n)" <6> 2021-06-18 15:53:29.773 +00:00 [INF] - Executing command: "Stop module SimulatedTemperatureSensor" <6> 2021-06-18 15:53:29.789 +00:00 [INF] - Executing command: "Start module SimulatedTemperatureSensor" <6> 2021-06-18 15:53:30.257 +00:00 [INF] - Executing command: "Saving SimulatedTemperatureSensor to store" <6> 2021-06-18 15:53:30.274 +00:00 [INF] - Executing command: "Command Group: (\n [Stop module edgeHub]\n [Start module edgeHub]\n [Saving edgeHub to store]\n)" <6> 2021-06-18 15:53:30.274 +00:00 [INF] - Executing command: "Stop module edgeHub" <6> 2021-06-18 15:53:30.277 +00:00 [INF] - Executing command: "Start module edgeHub" <6> 2021-06-18 15:53:31.096 +00:00 [INF] - Executing command: "Saving edgeHub to store" <6> 2021-06-18 15:53:31.099 +00:00 [INF] - Plan execution ended for deployment 2 <4> 2021-06-18 15:53:41.150 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 15:53:51.160 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <6> 2021-06-18 15:53:57.204 +00:00 [INF] - Edge agent attempting to connect to IoT Hub via Amqp_WebSocket_Only... <4> 2021-06-18 15:54:01.173 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 15:54:11.180 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 15:54:21.191 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 15:54:30.546 +00:00 [WRN] - Error creating a device-to-cloud connection System.AggregateException: One or more errors occurred. (Transient network error occurred, please retry.) (Transient network error occurred, please retry.) ---> Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.InitializeAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpIoTConnector.OpenConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.OpenSessionAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass26_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass37_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.CreateAndOpenSdkModuleClient(UpstreamProtocol upstreamProtocol, ConnectionStatusChangesHandler statusChangedHandler) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.IoTHub/ModuleClientProvider.cs:line 219 at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.<>c__DisplayClass19_1.<b__2>d.MoveNext() in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.IoTHub/ModuleClientProvider.cs:line 183 --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Edge.Util.Fallback.ExecuteAsync[T](Func`1[] options) --- End of inner exception stack trace --- ---> (Inner Exception #1) Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Net.WebSockets.WebSocketException (0x80004005): Unable to connect to the remote server ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.WebSockets.WebSocketHandle.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken, ClientWebSocketOptions options) at System.Net.WebSockets.WebSocketHandle.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken, ClientWebSocketOptions options) at System.Net.WebSockets.ClientWebSocket.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.CreateClientWebSocketAsync(Uri websocketUri, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.CreateClientWebSocketTransportAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.InitializeAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpIoTConnector.OpenConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.OpenSessionAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass26_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass37_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.CreateAndOpenSdkModuleClient(UpstreamProtocol upstreamProtocol, ConnectionStatusChangesHandler statusChangedHandler) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.IoTHub/ModuleClientProvider.cs:line 219 at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.<>c__DisplayClass19_1.<b__3>d.MoveNext() in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.IoTHub/ModuleClientProvider.cs:line 189 --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Edge.Util.Fallback.ExecuteAsync[T](Func`1[] options)<--- <6> 2021-06-18 15:54:30.564 +00:00 [INF] - Edge agent attempting to connect to IoT Hub via Amqp_Tcp_Only... <4> 2021-06-18 15:54:31.210 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 15:54:41.214 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 15:54:51.215 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 15:55:01.219 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <6> 2021-06-18 15:55:03.722 +00:00 [INF] - Edge agent attempting to connect to IoT Hub via Amqp_WebSocket_Only... <4> 2021-06-18 15:55:11.224 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 15:55:21.228 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 15:55:31.230 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 15:55:36.725 +00:00 [WRN] - Error creating a device-to-cloud connection System.AggregateException: One or more errors occurred. (Transient network error occurred, please retry.) (Transient network error occurred, please retry.) ---> Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.InitializeAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpIoTConnector.OpenConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.OpenSessionAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass26_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass37_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.CreateAndOpenSdkModuleClient(UpstreamProtocol upstreamProtocol, ConnectionStatusChangesHandler statusChangedHandler) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.IoTHub/ModuleClientProvider.cs:line 219 at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.<>c__DisplayClass19_1.<b__2>d.MoveNext() in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.IoTHub/ModuleClientProvider.cs:line 183 --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Edge.Util.Fallback.ExecuteAsync[T](Func`1[] options) --- End of inner exception stack trace --- ---> (Inner Exception #1) Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Net.WebSockets.WebSocketException (203): Unable to connect to the remote server ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.WebSockets.WebSocketHandle.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken, ClientWebSocketOptions options) at System.Net.WebSockets.WebSocketHandle.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken, ClientWebSocketOptions options) at System.Net.WebSockets.ClientWebSocket.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.CreateClientWebSocketAsync(Uri websocketUri, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.CreateClientWebSocketTransportAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.InitializeAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpIoTConnector.OpenConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.OpenSessionAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass26_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass37_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.CreateAndOpenSdkModuleClient(UpstreamProtocol upstreamProtocol, ConnectionStatusChangesHandler statusChangedHandler) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.IoTHub/ModuleClientProvider.cs:line 219 at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.<>c__DisplayClass19_1.<b__3>d.MoveNext() in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.IoTHub/ModuleClientProvider.cs:line 189 --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Edge.Util.Fallback.ExecuteAsync[T](Func`1[] options)<--- <4> 2021-06-18 15:55:41.235 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <6> 2021-06-18 15:55:43.480 +00:00 [INF] - Edge agent attempting to connect to IoT Hub via Amqp_Tcp_Only... <4> 2021-06-18 15:55:51.240 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 15:56:01.243 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 15:56:11.246 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <6> 2021-06-18 15:56:16.523 +00:00 [INF] - Edge agent attempting to connect to IoT Hub via Amqp_WebSocket_Only... <4> 2021-06-18 15:56:21.248 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 15:56:31.249 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 15:56:41.253 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 15:56:49.338 +00:00 [WRN] - Error creating a device-to-cloud connection System.AggregateException: One or more errors occurred. (Transient network error occurred, please retry.) (Transient network error occurred, please retry.) ---> Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.InitializeAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpIoTConnector.OpenConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.OpenSessionAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass26_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass37_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.CreateAndOpenSdkModuleClient(UpstreamProtocol upstreamProtocol, ConnectionStatusChangesHandler statusChangedHandler) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.IoTHub/ModuleClientProvider.cs:line 219 at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.<>c__DisplayClass19_1.<b__2>d.MoveNext() in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.IoTHub/ModuleClientProvider.cs:line 183 --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Edge.Util.Fallback.ExecuteAsync[T](Func`1[] options) --- End of inner exception stack trace --- ---> (Inner Exception #1) Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Net.WebSockets.WebSocketException (203): Unable to connect to the remote server ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.WebSockets.WebSocketHandle.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken, ClientWebSocketOptions options) at System.Net.WebSockets.WebSocketHandle.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken, ClientWebSocketOptions options) at System.Net.WebSockets.ClientWebSocket.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.CreateClientWebSocketAsync(Uri websocketUri, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.CreateClientWebSocketTransportAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.InitializeAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpIoTConnector.OpenConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.OpenSessionAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass26_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass37_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.CreateAndOpenSdkModuleClient(UpstreamProtocol upstreamProtocol, ConnectionStatusChangesHandler statusChangedHandler) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.IoTHub/ModuleClientProvider.cs:line 219 at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.<>c__DisplayClass19_1.<b__3>d.MoveNext() in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.IoTHub/ModuleClientProvider.cs:line 189 --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Edge.Util.Fallback.ExecuteAsync[T](Func`1[] options)<--- <4> 2021-06-18 15:56:51.257 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 15:57:01.259 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <6> 2021-06-18 15:57:03.045 +00:00 [INF] - Edge agent attempting to connect to IoT Hub via Amqp_Tcp_Only... <4> 2021-06-18 15:57:11.262 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 15:57:21.265 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 15:57:31.262 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <6> 2021-06-18 15:57:36.116 +00:00 [INF] - Edge agent attempting to connect to IoT Hub via Amqp_WebSocket_Only... <4> 2021-06-18 15:57:41.269 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 15:57:51.271 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 15:58:01.274 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 15:58:09.316 +00:00 [WRN] - Error creating a device-to-cloud connection System.AggregateException: One or more errors occurred. (Transient network error occurred, please retry.) (Transient network error occurred, please retry.) ---> Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.InitializeAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpIoTConnector.OpenConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.OpenSessionAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass26_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass37_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.CreateAndOpenSdkModuleClient(UpstreamProtocol upstreamProtocol, ConnectionStatusChangesHandler statusChangedHandler) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.IoTHub/ModuleClientProvider.cs:line 219 at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.<>c__DisplayClass19_1.<b__2>d.MoveNext() in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.IoTHub/ModuleClientProvider.cs:line 183 --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Edge.Util.Fallback.ExecuteAsync[T](Func`1[] options) --- End of inner exception stack trace --- ---> (Inner Exception #1) Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Net.WebSockets.WebSocketException (0x80004005): Unable to connect to the remote server ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.WebSockets.WebSocketHandle.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken, ClientWebSocketOptions options) at System.Net.WebSockets.WebSocketHandle.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken, ClientWebSocketOptions options) at System.Net.WebSockets.ClientWebSocket.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.CreateClientWebSocketAsync(Uri websocketUri, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.CreateClientWebSocketTransportAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.InitializeAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpIoTConnector.OpenConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.OpenSessionAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass26_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass37_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.CreateAndOpenSdkModuleClient(UpstreamProtocol upstreamProtocol, ConnectionStatusChangesHandler statusChangedHandler) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.IoTHub/ModuleClientProvider.cs:line 219 at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.<>c__DisplayClass19_1.<b__3>d.MoveNext() in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.IoTHub/ModuleClientProvider.cs:line 189 --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Edge.Util.Fallback.ExecuteAsync[T](Func`1[] options)<--- <4> 2021-06-18 15:58:11.277 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 15:58:21.282 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 15:58:31.283 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <6> 2021-06-18 15:58:37.157 +00:00 [INF] - Edge agent attempting to connect to IoT Hub via Amqp_Tcp_Only... <4> 2021-06-18 15:58:41.285 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 15:58:51.290 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 15:59:01.294 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <6> 2021-06-18 15:59:10.055 +00:00 [INF] - Edge agent attempting to connect to IoT Hub via Amqp_WebSocket_Only... <4> 2021-06-18 15:59:11.297 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 15:59:21.297 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 15:59:31.297 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 15:59:41.295 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 15:59:43.230 +00:00 [WRN] - Error creating a device-to-cloud connection System.AggregateException: One or more errors occurred. (Transient network error occurred, please retry.) (Transient network error occurred, please retry.) ---> Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.InitializeAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpIoTConnector.OpenConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.OpenSessionAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass26_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass37_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.CreateAndOpenSdkModuleClient(UpstreamProtocol upstreamProtocol, ConnectionStatusChangesHandler statusChangedHandler) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.IoTHub/ModuleClientProvider.cs:line 219 at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.<>c__DisplayClass19_1.<b__2>d.MoveNext() in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.IoTHub/ModuleClientProvider.cs:line 183 --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Edge.Util.Fallback.ExecuteAsync[T](Func`1[] options) --- End of inner exception stack trace --- ---> (Inner Exception #1) Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Net.WebSockets.WebSocketException (0x80004005): Unable to connect to the remote server ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.WebSockets.WebSocketHandle.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken, ClientWebSocketOptions options) at System.Net.WebSockets.WebSocketHandle.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken, ClientWebSocketOptions options) at System.Net.WebSockets.ClientWebSocket.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.CreateClientWebSocketAsync(Uri websocketUri, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.CreateClientWebSocketTransportAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.InitializeAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpIoTConnector.OpenConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.OpenSessionAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass26_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass37_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.CreateAndOpenSdkModuleClient(UpstreamProtocol upstreamProtocol, ConnectionStatusChangesHandler statusChangedHandler) in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.IoTHub/ModuleClientProvider.cs:line 219 at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.<>c__DisplayClass19_1.<b__3>d.MoveNext() in /home/vsts/work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.IoTHub/ModuleClientProvider.cs:line 189 --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Edge.Util.Fallback.ExecuteAsync[T](Func`1[] options)<--- <4> 2021-06-18 15:59:51.298 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 16:00:01.300 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 16:00:11.302 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 16:00:21.304 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 16:00:31.311 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <6> 2021-06-18 16:00:33.325 +00:00 [INF] - Edge agent attempting to connect to IoT Hub via Amqp_Tcp_Only... <4> 2021-06-18 16:00:41.313 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 16:00:51.319 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-06-18 16:01:01.319 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead ```
edge-hub logs ``` 2021-06-18 15:53:31 Starting Edge Hub Jun 18 15:53:31.028 INFO watchdog: Starting Watchdog Jun 18 15:53:31.028 INFO watchdog: Registering shutdown signal listener Jun 18 15:53:31.029 INFO watchdog: MQTT broker is disabled Jun 18 15:53:31.035 INFO watchdog::child: Launched Edge Hub process with pid 9 2021-06-18 15:53:31.137 +00:00 Edge Hub Main() <6> 2021-06-18 15:53:31.879 +00:00 [INF] - Installing certificates [CN=IoTEdgeGatewayCA.ca:07/18/2021 14:20:53],[CN=Azure_IoT_Hub_Intermediate_Cert_Test_Only:07/18/2021 14:20:32],[CN=Azure_IoT_Hub_CA_Cert_Test_Only:07/18/2021 14:20:32] to Root <6> 2021-06-18 15:53:31.910 +00:00 [INF] - Installing certificates [CN=IoTEdgeGatewayCA.ca:07/18/2021 14:20:53],[CN=Azure_IoT_Hub_Intermediate_Cert_Test_Only:07/18/2021 14:20:32],[CN=Azure_IoT_Hub_CA_Cert_Test_Only:07/18/2021 14:20:32],[CN=Azure_IoT_Hub_CA_Cert_Test_Only:07/18/2021 14:20:32] to Root <6> 2021-06-18 15:53:31.918 +00:00 [INF] - Enabling SSL protocols: Tls, Tls11, Tls12 <6> 2021-06-18 15:53:32.077 +00:00 [INF] - Experimental features configuration: {"Enabled":false,"DisableCloudSubscriptions":false,"DisableConnectivityCheck":false,"EnableMqttBroker":false} <6> 2021-06-18 15:53:32.640 +00:00 [INF] - Created persistent store at /tmp/edgeHub <6> 2021-06-18 15:53:32.671 +00:00 [INF] - Initializing Edge Hub <6> 2021-06-18 15:53:32.671 +00:00 [INF] - █████╗ ███████╗██╗ ██╗██████╗ ███████╗ ██╔══██╗╚══███╔╝██║ ██║██╔══██╗██╔════╝ ███████║ ███╔╝ ██║ ██║██████╔╝█████╗ ██╔══██║ ███╔╝ ██║ ██║██╔══██╗██╔══╝ ██║ ██║███████╗╚██████╔╝██║ ██║███████╗ ╚═╝ ╚═╝╚══════╝ ╚═════╝ ╚═╝ ╚═╝╚══════╝ ██╗ ██████╗ ████████╗ ███████╗██████╗ ██████╗ ███████╗ ██║██╔═══██╗╚══██╔══╝ ██╔════╝██╔══██╗██╔════╝ ██╔════╝ ██║██║ ██║ ██║ █████╗ ██║ ██║██║ ███╗█████╗ ██║██║ ██║ ██║ ██╔══╝ ██║ ██║██║ ██║██╔══╝ ██║╚██████╔╝ ██║ ███████╗██████╔╝╚██████╔╝███████╗ ╚═╝ ╚═════╝ ╚═╝ ╚══════╝╚═════╝ ╚═════╝ ╚══════╝ <6> 2021-06-18 15:53:32.673 +00:00 [INF] - Version - 1.2.1.43175312 (f0e7486e3fd527515999b9449313e55b733516c9) <6> 2021-06-18 15:53:32.673 +00:00 [INF] - OptimizeForPerformance=True <6> 2021-06-18 15:53:32.673 +00:00 [INF] - MessageAckTimeoutSecs=30 <6> 2021-06-18 15:53:32.673 +00:00 [INF] - Loaded server certificate with expiration date of "2021-07-18T14:20:53.0000000+00:00" <6> 2021-06-18 15:53:32.679 +00:00 [INF] - Using Asp Net server for metrics <6> 2021-06-18 15:53:32.712 +00:00 [INF] - Created new message store <6> 2021-06-18 15:53:32.712 +00:00 [INF] - Started task to cleanup processed and stale messages <6> 2021-06-18 15:53:32.762 +00:00 [INF] - Created DeviceConnectivityManager with connected check frequency 00:05:00 and disconnected check frequency 00:02:00 <6> 2021-06-18 15:53:32.836 +00:00 [INF] - Add node: di-10 <6> 2021-06-18 15:53:32.837 +00:00 [INF] - Add node: di-10/$edgeAgent <6> 2021-06-18 15:53:32.837 +00:00 [INF] - Add node: di-10/$edgeHub <6> 2021-06-18 15:53:32.837 +00:00 [INF] - Add node: di-10/SimulatedTemperatureSensor <6> 2021-06-18 15:53:32.838 +00:00 [INF] - Add node: sensor-th-0084 <6> 2021-06-18 15:53:32.843 +00:00 [INF] - Created device scope identities cache <6> 2021-06-18 15:53:32.849 +00:00 [INF] - Starting refresh of device scope identities cache <6> 2021-06-18 15:53:32.853 +00:00 [INF] - Initialized storing twin manager <6> 2021-06-18 15:53:32.859 +00:00 [INF] - Updated node: di-10/$edgeHub <6> 2021-06-18 15:53:32.859 +00:00 [INF] - Initializing configuration <6> 2021-06-18 15:53:32.869 +00:00 [INF] - New device connection for device di-10/$edgeHub <6> 2021-06-18 15:53:32.875 +00:00 [INF] - Client di-10/$edgeHub connected to edgeHub, processing existing subscriptions. <6> 2021-06-18 15:53:32.915 +00:00 [INF] - Attempting to connect to IoT Hub for client di-10/$edgeHub via AMQP... <6> 2021-06-18 15:53:32.978 +00:00 [INF] - Processing pending subscriptions for di-10/$edgeHub <6> 2021-06-18 15:53:33.104 +00:00 [INF] - Experimental features configuration: {"Enabled":false,"DisableCloudSubscriptions":false,"DisableConnectivityCheck":false,"EnableMqttBroker":false} <6> 2021-06-18 15:53:33.116 +00:00 [INF] - Starting timer to authenticate connections with a period of 300 seconds <6> 2021-06-18 15:53:33.160 +00:00 [INF] - Scheduling server certificate renewal for "2021-07-18T14:18:23.0000729Z". <6> 2021-06-18 15:53:33.305 +00:00 [INF] - Created cloud endpoint iothub with max batch size 10 and fan-out factor of 10. <6> 2021-06-18 15:53:33.419 +00:00 [INF] - Set the following 2 route(s) in edge hub <6> 2021-06-18 15:53:33.419 +00:00 [INF] - route: FROM /messages/* INTO $upstream <6> 2021-06-18 15:53:33.420 +00:00 [INF] - SimulatedTemperatureSensorToIoTHub: FROM /messages/modules/SimulatedTemperatureSensor/* INTO $upstream <6> 2021-06-18 15:53:33.420 +00:00 [INF] - Updated message store TTL to 7200 seconds <6> 2021-06-18 15:53:33.420 +00:00 [INF] - Updated the edge hub store and forward configuration <6> 2021-06-18 15:53:33.432 +00:00 [INF] - Started operation Get EdgeHub config <6> 2021-06-18 15:53:33.432 +00:00 [INF] - Initialized edge hub configuration <6> 2021-06-18 15:53:33.432 +00:00 [INF] - Starting protocol heads - (MQTT, AMQP, HTTP) <6> 2021-06-18 15:53:33.435 +00:00 [INF] - Starting MQTT head <6> 2021-06-18 15:53:33.453 +00:00 [INF] - Initializing TLS endpoint on port 8883 for MQTT head. <6> 2021-06-18 15:53:33.486 +00:00 [INF] - Starting AMQP head <6> 2021-06-18 15:53:33.495 +00:00 [INF] - Started MQTT head <6> 2021-06-18 15:53:33.560 +00:00 [INF] - Started AMQP head <6> 2021-06-18 15:53:33.561 +00:00 [INF] - Starting HTTP head <4> 2021-06-18 15:53:33.702 +00:00 [WRN] - Overriding address(es) '"http://+:80"'. Binding to endpoints defined in "UseKestrel()" instead. <6> 2021-06-18 15:53:33.709 +00:00 [INF] - Started HTTP head <4> 2021-06-18 15:53:53.832 +00:00 [WRN] - Error creating cloud connection for client di-10/$edgeHub Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.InitializeAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpIoTConnector.OpenConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.OpenSessionAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass26_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass37_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout) in /home/vsts/work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 143 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 55 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 186 --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 145 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 173 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 62 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 132 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 107 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, String productInfo, Option`1 modelId) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 96 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 279 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 250 <6> 2021-06-18 15:53:53.859 +00:00 [INF] - Error getting cloud connection for device di-10/$edgeHub Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.InitializeAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpIoTConnector.OpenConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.OpenSessionAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass26_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass37_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout) in /home/vsts/work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 143 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 55 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 186 --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 145 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 173 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 62 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 132 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 107 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, String productInfo, Option`1 modelId) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 96 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 279 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 250 <6> 2021-06-18 15:53:53.859 +00:00 [INF] - Error getting cloud connection for device di-10/$edgeHub Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.InitializeAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpIoTConnector.OpenConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.OpenSessionAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass26_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass37_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout) in /home/vsts/work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 143 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 55 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 186 --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 145 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 173 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 62 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 132 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 107 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, String productInfo, Option`1 modelId) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 96 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 279 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 250 <6> 2021-06-18 15:53:53.862 +00:00 [INF] - Error getting cloud connection for device di-10/$edgeHub Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.InitializeAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpIoTConnector.OpenConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.OpenSessionAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass26_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass37_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout) in /home/vsts/work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 143 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 55 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 186 --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 145 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 173 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 62 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 132 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 107 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, String productInfo, Option`1 modelId) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 96 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 279 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 250 <6> 2021-06-18 15:53:53.862 +00:00 [INF] - Processing pending subscriptions for di-10/$edgeHub, but no cloud proxy was found <6> 2021-06-18 15:53:53.866 +00:00 [INF] - Error getting cloud connection for device di-10/$edgeHub Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.InitializeAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpIoTConnector.OpenConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.OpenSessionAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass26_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass37_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout) in /home/vsts/work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 143 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 55 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 186 --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 145 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 173 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 62 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 132 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 107 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, String productInfo, Option`1 modelId) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 96 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 279 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 250 <4> 2021-06-18 15:53:53.870 +00:00 [WRN] - Updating reported properties failed di-10/$edgeHub <6> 2021-06-18 15:53:53.875 +00:00 [INF] - Processing pending subscriptions for di-10/$edgeHub <6> 2021-06-18 15:53:53.984 +00:00 [INF] - Obtained edge hub config from module twin <6> 2021-06-18 15:53:53.995 +00:00 [INF] - Set the following 2 route(s) in edge hub <6> 2021-06-18 15:53:53.995 +00:00 [INF] - route: FROM /messages/* INTO $upstream <6> 2021-06-18 15:53:53.995 +00:00 [INF] - SimulatedTemperatureSensorToIoTHub: FROM /messages/modules/SimulatedTemperatureSensor/* INTO $upstream <6> 2021-06-18 15:53:53.995 +00:00 [INF] - Updated message store TTL to 7200 seconds <6> 2021-06-18 15:53:53.995 +00:00 [INF] - Updated the edge hub store and forward configuration <6> 2021-06-18 15:53:58.934 +00:00 [INF] - Attempting to connect to IoT Hub for client di-10/$edgeHub via AMQP... <4> 2021-06-18 15:54:19.675 +00:00 [WRN] - Error creating cloud connection for client di-10/$edgeHub Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.InitializeAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpIoTConnector.OpenConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.OpenSessionAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass26_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass37_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout) in /home/vsts/work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 143 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 55 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 186 --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 145 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 173 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 62 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 132 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 107 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, String productInfo, Option`1 modelId) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 96 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 279 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 250 <6> 2021-06-18 15:54:19.680 +00:00 [INF] - Error getting cloud connection for device di-10/$edgeHub Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.InitializeAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpIoTConnector.OpenConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.OpenSessionAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass26_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass37_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout) in /home/vsts/work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 143 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 55 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 186 --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 145 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 173 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 62 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 132 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 107 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, String productInfo, Option`1 modelId) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 96 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 279 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 250 <4> 2021-06-18 15:54:19.682 +00:00 [WRN] - Updating reported properties failed di-10/$edgeHub <4> 2021-06-18 15:55:12.892 +00:00 [WRN] - Encountered an error while refreshing the device scope identities cache. Will retry the operation in some time... <6> 2021-06-18 15:55:12.893 +00:00 [INF] - Done refreshing device scope identities cache. Waiting for 60 minutes. <6> 2021-06-18 15:58:32.854 +00:00 [INF] - Attempting to connect to IoT Hub for client di-10/$edgeHub via AMQP... <6> 2021-06-18 15:58:33.119 +00:00 [INF] - Entering periodic task to reauthenticate connected clients <4> 2021-06-18 15:58:53.598 +00:00 [WRN] - Error creating cloud connection for client di-10/$edgeHub Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.InitializeAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpIoTConnector.OpenConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.OpenSessionAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass26_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass37_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout) in /home/vsts/work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 143 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 55 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 186 --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 145 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 173 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 62 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 132 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 107 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, String productInfo, Option`1 modelId) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 96 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 279 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 250 <6> 2021-06-18 15:58:53.599 +00:00 [INF] - Error getting cloud connection for device di-10/$edgeHub Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.InitializeAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpIoTConnector.OpenConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.OpenSessionAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionIsOpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass26_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass37_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Util.TaskEx.TimeoutAfter(Task task, TimeSpan timeout) in /home/vsts/work/1/s/edge-util/src/Microsoft.Azure.Devices.Edge.Util/TaskEx.cs:line 143 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ModuleClientWrapper.OpenAsync() in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ModuleClientWrapper.cs:line 55 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.<>c__DisplayClass30_0.<b__0>d.MoveNext() in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 186 --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 145 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.InvokeFunc[T](Func`1 func, String operation, Boolean useForConnectivityCheck) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 173 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.ConnectivityAwareClient.OpenAsync() in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/ConnectivityAwareClient.cs:line 62 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.ConnectToIoTHub(ITokenProvider newTokenProvider) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 132 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.CreateNewCloudProxyAsync(ITokenProvider newTokenProvider) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 107 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnection.Create(IIdentity identity, Action`2 connectionStatusChangedHandler, ITransportSettings[] transportSettings, IMessageConverterProvider messageConverterProvider, IClientProvider clientProvider, ICloudListener cloudListener, ITokenProvider tokenProvider, TimeSpan idleTimeout, Boolean closeOnIdleTimeout, TimeSpan operationTimeout, String productInfo, Option`1 modelId) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnection.cs:line 96 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.TryCreateCloudConnectionFromServiceIdentity(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshOutOfDateCache, CloudListener cloudListener, String authChain) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 279 at Microsoft.Azure.Devices.Edge.Hub.CloudProxy.CloudConnectionProvider.ConnectInternalWithDeviceStateTracking(IIdentity identity, Action`2 connectionStatusChangedHandler, Boolean refreshCachedIdentity) in /home/vsts/work/1/s/edge-hub/core/src/Microsoft.Azure.Devices.Edge.Hub.CloudProxy/CloudConnectionProvider.cs:line 250 ```

Additional Information

I think the creation of the module client hangs. I'm missing the following log line: [Information]: Successfully initialized module client of transport type [Amqp_Tcp_Only].

SimulatedTemperatureSensor logs:

[2021-06-18 15:53:30 : Starting Module
SimulatedTemperatureSensor Main() started.
Initializing simulated temperature sensor to send 500 messages, at an interval of 5 seconds.
To change this, set the environment variable MessageCount to the number of messages that should be sent (set it to -1 to send unlimited messages).
[Information]: Trying to initialize module client using transport type [Amqp_Tcp_Only].
dmolokanov commented 3 years ago

Hello @shaeussler ! It looks like the edgeHub cannot establish connection to the iothub via AMQP for some reason:

<4> 2021-06-18 15:58:53.598 +00:00 [WRN] - Error creating cloud connection for client di-10/$edgeHub
Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry.
 ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable

Same for the edgeAgent 

<4> 2021-06-18 15:54:30.546 +00:00 [WRN] - Error creating a device-to-cloud connection
System.AggregateException: One or more errors occurred. (Transient network error occurred, please retry.) (Transient network error occurred, please retry.)
 ---> Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry.
 ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable

It makes me think something off with the firewall rules: the code from the inside module cannot access iothub. I think you can start investigating from there.

dmolokanov commented 3 years ago

It looks like we have another issue #5125 with similar symptoms

shaeussler commented 3 years ago

I do not run a firewall on the Ubuntu VM. I blocked all outbound ports in the network security group of the virtual network to simulate the offline case: grafik

dmolokanov commented 3 years ago

Thanks for clarification! This behavior may be expected if you use FQDN/Public IP as a hostname for iotedge. From edge module it'll try to connect to a edgehub using FQDN will require to go outside of the machine. But since you have firewall rules to block everything it will fail even connections on the same machine.

shaeussler commented 3 years ago

Hostname of the iotege host is "xx-yy-ubuntu1804". On the host in offline mode I can ping the host: ping xx-yy-ubuntu1804 returns 64 bytes from xx-yy-ubuntu1804.internal.cloudapp.net (10.1.1.4): icmp_seq=1 ttl=64 time=0.026 ms

dmolokanov commented 3 years ago

Could you please try to run vanilla ubuntu container and try to connect to edgehub using openssl command?

# this should run a new container in the same network as edgeHub
$ docker run -it --rm --network container:edgeHub ubuntu

# inside the container try to connect to edgeHub amqp port
$ openssl s_client -connect xx-yy-ubuntu1804:5671 -showcerts
shaeussler commented 3 years ago

I set the edge to online mode and run the following command: $ docker run -it --rm --network container:edgeHub ubuntu Inside the container I run "apt-get update && apt-get upgrade" and "apt-get install openssl". Here is the result of the command "openssl s_client -connect xx-yy-ubuntu1804:5671 -showcerts"

Click here ``` openssl s_client -connect xx-yy-ubuntu1804:5671 -showcerts CONNECTED(00000003) Can't use SSL_get_servername depth=2 CN = Azure_IoT_Hub_Intermediate_Cert_Test_Only verify error:num=20:unable to get local issuer certificate verify return:1 depth=1 CN = IoTEdgeGatewayCA.ca verify return:1 depth=0 CN = xx-yy-ubuntu1804 verify return:1 --- Certificate chain 0 s:CN = xx-yy-ubuntu1804 i:CN = IoTEdgeGatewayCA.ca -----BEGIN CERTIFICATE----- MIID8TCCAdmgAwIBAgIBADANBgkqhkiG9w0BAQsFADAeMRwwGgYDVQQDDBNJb1RF ZGdlR2F0ZXdheUNBLmNhMB4XDTIxMDYyODA1NDY1MFoXDTIxMDcxODE0MjA1M1ow GzEZMBcGA1UEAwwQZGktMTAtdWJ1bnR1MTgwNDCCASIwDQYJKoZIhvcNAQEBBQAD ggEPADCCAQoCggEBAMqFRh3xHkEu2XYZ+2MGK40V6vvBpsdl5Y3hXg8F0ifAEYgn rhgzwQTAyQea4QaH8MxJO/VmaxT32O6T9ZLAuJ5H2DNVDCzvUSqNA7ojVEHyr0dg zT6wcfnanHBoqVwQziVCiSrb0DT0roKPRF/R9k9WTCfNC2k20wkVGOgxZkriBGnu b/tcpEwh0ieC7wLgVNKf+wKWOASOdM5ONN98kE/meFVFj7T8nq6QRvigatppWj8m TGtz439HMY+TY1Jfh1YtBqn3SgUuqfQtv+zlA7AzbPkqwi6/HtkIAC65IihMfz1/ J01r5BYWBC6qN76FAHGupFHsJhdFRGp/wS9j2d0CAwEAAaM9MDswEwYDVR0lBAww CgYIKwYBBQUHAwEwJAYDVR0RBB0wG4IHZWRnZWh1YoIQZGktMTAtdWJ1bnR1MTgw NDANBgkqhkiG9w0BAQsFAAOCAgEAtwf2kYeTfpRPpmsLa1ZAWSRSvtPwrB4Af1PW WOouR32fSLwNossdKvHWKj/bT3jnhzApAQHj6ojQB+vSZwsTZyT8bn91qod7JyTA tPQWYJwXB4lWvza97amF4A9OGa+UQrNmmeIjMQ7XkELKLDu/oChZx5dxdrHinzi0 JVKI8hc9i8AM6y61iX4BJUZiDV0fOo+eD79lLXG4QUEb5Y0uJFLODBYDOJFwNr44 aoFUUQ7A7Mk8Fw/fVzRJbCBsyzD/IQu8fhaV7ovEvSVTAzaKsncRuIcvY4pkvxHf uKDNods+urRzrItbMyPX1S8hsPQN3gLzkzdX/WBKwCBEiqUBW3tmMlAiVEh3MV5c /tvJJQxRL4dtTzRTQYVxNCx/hg8u7e8vZxVeoxSyXq51MsV7WOdYcJxDDmEyycg9 /0Wa9JsUCQgEaVu53EDGjJtYhBoFjidFUphicau7FSOD6XxMqtnYUHbuiZDI2pC9 wiFY/g9O4H1WfHZtAPBjfeXQKHfkojIZe1u6nzQIi8gCLnFH0ARjj4Ea150PLwhM H6d0+XUK3+aA+4ghFBQXMTxnGXuW/hFyMNAYFI1DsO88JLkkHZtXBGAjqISs6WQI Ph9liZbHYYk/duDM0gsQOcOBMF9BO04Cm3yYo7zZ9+7Gt3+YbduGtLd6TaFzlcco QNLcYnk= -----END CERTIFICATE----- 1 s:CN = IoTEdgeGatewayCA.ca i:CN = Azure_IoT_Hub_Intermediate_Cert_Test_Only -----BEGIN CERTIFICATE----- MIIFZzCCA0+gAwIBAgICEAEwDQYJKoZIhvcNAQELBQAwNDEyMDAGA1UEAwwpQXp1 cmVfSW9UX0h1Yl9JbnRlcm1lZGlhdGVfQ2VydF9UZXN0X09ubHkwHhcNMjEwNjE4 MTQyMDUzWhcNMjEwNzE4MTQyMDUzWjAeMRwwGgYDVQQDDBNJb1RFZGdlR2F0ZXdh eUNBLmNhMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA11gE7qTt4LqF OkmOgJeoQA74UPmvkIi58gpnAl9FbIW9GS1QWVZAKTdwSkK/D0+Askdsl1acw4uS hvO4TLW/okvKUUrXqkdQbr4sQC15jO543YGnEVRCESoOf2/Nwu1oZfQ92pzsKkK5 wXd5lzO68Stodv3WwFeN5VCpUSc+/EXerCwSVGtjSy1JqYQ5IZrfdQqmv9cNFk1V 3GesaiyccEdgpkcJy2h35hvrh9a9ynZGuGAX8+o9XydjRqDC+BCb3EsWTJYwVr5l 0/B04zhIVVgHGW/KbzSwQBHpZp9khG1GBzhxj5MDBLIdEaNQfNY0Er9jVMD9IqGS JAhk+HskrRxNAKy3htT956PK2s1GwBH5F7Ebc8BDC0FzjsGi8Moo0wbi2ej6vdu2 fTt+EBzl/kz5Q51gnKa5pmZ1Zg/Z+//SEQU4eMGh8veJzxv2QHHJz+QY59PjNHML wr1i4ywfyhZsD9nQ/Tojfa90gX4NjW9NZ5ZJtiJshAmY48sN5L3OnAqBQ2jlFUlI kEtBs/wUXp1nZvrWYVSHGmDRH/duZcv5DdzUIHZrzfWSi24d55xXKbSZrshi2LvU PtLnX4q+km0HmpeX/hjArMU8B8NjyvZWYl2g28wJNTfb2S3SReFGQqufLCssE43D UUrS/bkFV2OYwcDpUcGYkZkgtXWGlvUCAwEAAaOBmDCBlTAdBgNVHQ4EFgQU7RTW 9XEIqmw8DV+LAOTtrX13744wUwYDVR0jBEwwSoAU/za5AJO3639oMJj7AePXbh6S pomhLqQsMCoxKDAmBgNVBAMMH0F6dXJlX0lvVF9IdWJfQ0FfQ2VydF9UZXN0X09u bHmCAhAAMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3 DQEBCwUAA4ICAQB75xeVzuwW8eNBJ+Ztn5PNJbi+OhIGpvWX+aigzI2JQd72BAUR gv8CAeNbib0JYlB/LVZ0TMJRbyJV8BMG+Nen5bl+wehPq1ce0J5QT7cILHL7rpKZ GKbyI3EOuhFG/xc592N72FTz7DYIzYgG7tj/WuidbC7pCNNZfyNerbOkn1It4iae eDwzgrNFPusj0tHFFtIH1nJKxBdjDVlLy6cFHpZlTw5S3qMMg9npgwXCZZvmSYZE DKWFxikl9MIXkvTIHiNxF/C5Ia1WiJeAjTnVwdkAMKrU3s1QKyOlhiTz1rM3I+AQ 89mGKEbVJrE8j1ylbJECCiiE4EnIexTATd6Yw8Fmhl9CGDk7zU+jNX0b2E7G8/eC H2VKoyZWB28q0tfeW607vwephCMDJ2eBQ/UBuBsLON7fakGLRIhfrrQlyFYdEmMV b7ML85mBClVtl41dsmln4RevIJxxOxi71oaJEs0acOrPpZfY+0yw6CvhSVEFAlfD oNl963R/tTSB33stV4wsC+Bfxzq1djlFhFXTlfhPrGPEN9ndLB79rlGOlbtBxR9x 6uoct6k/JBUvmSyQLSOvw5V6fTVd9UPC8KJBsTEXqjcTLLsVPjk+AoIWOKfiOQLQ yIdKYUcLFgRaphS3sOn5ImVmRys++tcuUaW97s+fdOluxr/T25WLovB3tw== -----END CERTIFICATE----- 2 s:CN = Azure_IoT_Hub_Intermediate_Cert_Test_Only i:CN = Azure_IoT_Hub_CA_Cert_Test_Only -----BEGIN CERTIFICATE----- MIIFhTCCA22gAwIBAgICEAAwDQYJKoZIhvcNAQELBQAwKjEoMCYGA1UEAwwfQXp1 cmVfSW9UX0h1Yl9DQV9DZXJ0X1Rlc3RfT25seTAeFw0yMTA2MTgxNDIwMzJaFw0y MTA3MTgxNDIwMzJaMDQxMjAwBgNVBAMMKUF6dXJlX0lvVF9IdWJfSW50ZXJtZWRp YXRlX0NlcnRfVGVzdF9Pbmx5MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKC AgEAqYklnKwrr8GMlwtAdyr025B7wo/g5j4fjethQUloOkCdsAB48cI2sOekAeMF ApszuAc3l/BVV4/9JeFlS/mEokTTQzi9KxZst57Oq3njFORaKdePTwRrERuag7CZ RAVQEJTEmJFnvVNLBxAl61eOV5j7/rRp761pes8tQmaMaArg3mFhdHNATbrErsNE F3VoFf0WftEjn1P9p04L9tENimmfw+uozKDoloEC1l7jGbVfHirk5m0UGbDtdsMU 707bVsf9fqQ7d9CMxbGwdo+WDjSRFE0yAYdQG9KQGELkOMP41gLR6BxJdVXR/6P4 OqTIcNPL4j55aOGJJ4dORA3umDymP1ZPnpejQOu81cqNqPClESQMQUVD8mCvu1Mf tqOiSJmnrGpXQnDJnFsyqMMY3/dQveD/dxH/zDMHxqdot5UIxvU2F4/0gR0kD40S 5k619S2T+FwpL6ip/RjWsRYDsrX0FMrkFBkMSuYGI9TuOYLar65apgNqXmCX7kdL F3a9OndTuuSQMXzSM6q+6vOjufEUEhUmraxuJ/ZCZND3lWQuFohQrFsgyjINCU1+ lQAr0ye/FwsRDP/O8hHrZp58KjQOzDJPC6Nh8uAS+bRaU5S1Xf1fRLv6hfte/Jyf QTVYVDaTm+N21dz35o0J7UBSO/6B/RMNAlrAfWiNnbbRtOUCAwEAAaOBqjCBpzAd BgNVHQ4EFgQU/za5AJO3639oMJj7AePXbh6SpokwZQYDVR0jBF4wXIAUdPm37OCg qMgW6LmO7siQjZiTfR2hLqQsMCoxKDAmBgNVBAMMH0F6dXJlX0lvVF9IdWJfQ0Ff Q2VydF9UZXN0X09ubHmCFEqXe9Y3h7ib1H3Yufh3gcTI0eu9MA8GA1UdEwEB/wQF MAMBAf8wDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4ICAQBdFGTGOHd5 e+SKU9MXfqM060CbQ/YHxkdf2NbQZdsRQHryWK3e2XSX/lMCHQE2NYj7X6oipiSX t9EadthGh34ik+6Yh0JnwsrNCUqT2feVD0I9qYLexW8WgfT2VqJTTSsysl2SqXL1 PwlrAGxFZlKwQO206l7DJg3YdLczqp9AQtOTO/u67GYq9TsMyYo60msqxtrWSXtk NLHqBITthuxR4r28me8C+jmnluWrgzv6zb2jcOuXBxlkR/NHISZ5y80hprifHXdM W9j8vipbZlAbBA34GRhdX2ePH8dq2oZkObryWQPeGeKDXziIHFKHre7irLp3VhyC Qjwxlu6l2irCMxH6S0YKEJS30VGxeFOlh3tkNsjAFnMwEyAasxQ35nvAhzWBCpMc bzs64LZUeA9pYQdkeD5kQy9Qom6Drsu1u96fkLWnLp6lr69fFd2W5kcM39EMBu0A 6ofnmLfw1df9emgT6VqavSBCMQ33b3n59or6KWJUJ+oNUwi9MR1CD36g6PgNZKMo qa+6AGzu19nUu0KLXbiMfKUeqqi7k2BLpJuTUQDaNm4+pa5xnrjzgqdBmxpew6F9 0jJCtbbPFNYiw+nR2NWy9yQLmOS3Q/ITrzYwNraICzrJitkwLYwZ8gThEQxxuTjR 7NP5JfEEoHXxeH0w/FsqQZKtEiuvaNgwHQ== -----END CERTIFICATE----- --- Server certificate subject=CN = xx-yy-ubuntu1804 issuer=CN = IoTEdgeGatewayCA.ca --- No client certificate CA names sent Client Certificate Types: RSA sign, DSA sign, ECDSA sign Requested Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:ECDSA+SHA1:RSA+SHA224:RSA+SHA1:DSA+SHA224:DSA+SHA1:DSA+SHA256:DSA+SHA384:DSA+SHA512 Shared Requested Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:RSA+SHA224:DSA+SHA224:DSA+SHA256:DSA+SHA384:DSA+SHA512 Peer signing digest: SHA256 Peer signature type: RSA-PSS Server Temp Key: X25519, 253 bits --- SSL handshake has read 4511 bytes and written 388 bytes Verification error: unable to get local issuer certificate --- New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384 Server public key is 2048 bit Secure Renegotiation IS supported Compression: NONE Expansion: NONE No ALPN negotiated SSL-Session: Protocol : TLSv1.2 Cipher : ECDHE-RSA-AES256-GCM-SHA384 Session-ID: 63B88FFAD5C5066D8FF5D2CAF65DB44699E9E26B8EC167790FD5C15B6C1088EA Session-ID-ctx: Master-Key: 480374FE4EECD0059CCA6879DD2889436B610D5D372AC84AFE82F556AE6B108FB91DDD9ABD69BAE3287655E728E6F5D3 PSK identity: None PSK identity hint: None SRP username: None TLS session ticket lifetime hint: 7200 (seconds) TLS session ticket: 0000 - 6a 33 72 5d 86 41 4d 02-21 40 54 ea 02 cd c6 a7 j3r].AM.!@T..... 0010 - fe 65 bf 02 39 12 2a 3a-88 d3 b6 72 c9 2b d8 f9 .e..9.*:...r.+.. 0020 - fc 94 d0 59 48 75 70 a9-65 d0 2b 20 bf 80 bb 93 ...YHup.e.+ .... 0030 - 5a 42 96 80 8d ea f4 07-18 bb df 9d 12 c6 c8 5b ZB.............[ 0040 - ce 4f fc e3 be 13 f9 8a-b9 96 ce 78 48 36 30 3a .O.........xH60: 0050 - 0b cd f6 26 bb 6e 0b 1c-a4 c3 d1 5d fe 35 10 2b ...&.n.....].5.+ 0060 - 74 6d 7e 92 b4 d5 e3 09-ca fe 82 23 a8 28 90 cf tm~........#.(.. 0070 - 47 05 d1 35 3e dc 8e 1b-2c 34 d2 51 83 02 32 d2 G..5>...,4.Q..2. 0080 - d0 de 75 9c fd 3d 61 05-99 f2 20 a3 59 0b ce ce ..u..=a... .Y... 0090 - 39 7c b2 c3 49 d6 a7 2e-fa c8 64 7e e5 f0 5d 81 9|..I.....d~..]. Start Time: 1624860551 Timeout : 7200 (sec) Verify return code: 20 (unable to get local issuer certificate) Extended master secret: yes ```

I saved the ubuntu container into a docker image, switched the edge into offline mode and rebooted the ubuntu VM xx-yy-ubuntu1804. Inside the container created from the saved ubuntu image I run the openssl command again. Here is the output:

Click here ``` openssl s_client -connect xx-yy-ubuntu1804:5671 -showcerts CONNECTED(00000003) Can't use SSL_get_servername depth=2 CN = Azure_IoT_Hub_Intermediate_Cert_Test_Only verify error:num=20:unable to get local issuer certificate verify return:1 depth=1 CN = IoTEdgeGatewayCA.ca verify return:1 depth=0 CN = xx-yy-ubuntu1804 verify return:1 --- Certificate chain 0 s:CN = xx-yy-ubuntu1804 i:CN = IoTEdgeGatewayCA.ca -----BEGIN CERTIFICATE----- MIID8TCCAdmgAwIBAgIBADANBgkqhkiG9w0BAQsFADAeMRwwGgYDVQQDDBNJb1RF ZGdlR2F0ZXdheUNBLmNhMB4XDTIxMDYyODA2MzQyM1oXDTIxMDcxODE0MjA1M1ow GzEZMBcGA1UEAwwQZGktMTAtdWJ1bnR1MTgwNDCCASIwDQYJKoZIhvcNAQEBBQAD ggEPADCCAQoCggEBAKb1dGZipgfDZ10VRXtY59VE71TJvDPbwpeq2geNU4cH1WjV 8PKKp73dTdGRrKxT4Tyjd7+m786GcHUQm6vchf29L08EAsUVox4wQBIZdrkI5dKj lgO37wN//5BzlbOLYyC5zWof6jNp2EH1YetGQSEwcJhQSc5a1GWNIeG8KkfFxkaj 0GT7T7UMYJ3N4i/He/9ThKzPuqQM10zQNcaBGuskwaBiwB6xdMbQAGzUHhJ7wxEy XdUlsH0gse9LlYWY5FypFfV6CcciYI1ru/ii8DLcKXUvVKCLa2Exe9G2gwQN5T8d dZbO6WiySmdMcepCaJRmrE5qNO8nuyHKCzZOygMCAwEAAaM9MDswEwYDVR0lBAww CgYIKwYBBQUHAwEwJAYDVR0RBB0wG4IHZWRnZWh1YoIQZGktMTAtdWJ1bnR1MTgw NDANBgkqhkiG9w0BAQsFAAOCAgEAGp6eCjYfOZLRoXAIfoOZTxJbgPX7gFZIDYAY ob2wft6DW1Ak7c2kCB20AydoqB7hzj0JV3D+fIyhbXP1/RwqtbooB5LE095s8v97 ktbxy4+KgjtJhkgXbSzdnxpcybgaa+hKYGlRFwDayRgIEiPew2mreBfyYIWkcxNY +kPvD9zIDMs7oSr2cOXHSLgIRqg4No32wfgkKIN6QVjb6z/Qr/pQwQpplbXd3Q5t 8NoYYBDwvLW9surMRPM3eCVhmvXMigLzBIwGvux606Yd0M42zSeonHnyTxdF3oGR I6VvXspj+ei3WPOJfiunv3uV4hhwSCalV/ecRGEbPhTzMd2iHr31NqiP666ZY0LS B6CIsMYGKKR62A0Q66zY4CHWlYMLAvEGeZGdgWijFLjpwGNUTt+5W4zsNS+h+DY7 AOIpQoIMlcuUwrmv+i15Xs2QMTNkolUVQPUF0M0MeA5bp5Y8cMiuF8J2VWt6FEHC mldMTSBMCkmRQad3zASHhWUpzdhnec73sjvLfmYuOMODJWp0EXCEVFL5D4xRyE1I eBCO7E1Qrzn05NhCp/xz/GrG44rDZL9MibeU8/xO8n8wz51tDhDGYWAQWrkjzWQP b6R/M0wazT01xP3IUC+7ZTzxaXRFViB1Kw4od1NZy996BQPIcV+/dLaYZou2J7R+ sbXPpK0= -----END CERTIFICATE----- 1 s:CN = IoTEdgeGatewayCA.ca i:CN = Azure_IoT_Hub_Intermediate_Cert_Test_Only -----BEGIN CERTIFICATE----- MIIFZzCCA0+gAwIBAgICEAEwDQYJKoZIhvcNAQELBQAwNDEyMDAGA1UEAwwpQXp1 cmVfSW9UX0h1Yl9JbnRlcm1lZGlhdGVfQ2VydF9UZXN0X09ubHkwHhcNMjEwNjE4 MTQyMDUzWhcNMjEwNzE4MTQyMDUzWjAeMRwwGgYDVQQDDBNJb1RFZGdlR2F0ZXdh eUNBLmNhMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA11gE7qTt4LqF OkmOgJeoQA74UPmvkIi58gpnAl9FbIW9GS1QWVZAKTdwSkK/D0+Askdsl1acw4uS hvO4TLW/okvKUUrXqkdQbr4sQC15jO543YGnEVRCESoOf2/Nwu1oZfQ92pzsKkK5 wXd5lzO68Stodv3WwFeN5VCpUSc+/EXerCwSVGtjSy1JqYQ5IZrfdQqmv9cNFk1V 3GesaiyccEdgpkcJy2h35hvrh9a9ynZGuGAX8+o9XydjRqDC+BCb3EsWTJYwVr5l 0/B04zhIVVgHGW/KbzSwQBHpZp9khG1GBzhxj5MDBLIdEaNQfNY0Er9jVMD9IqGS JAhk+HskrRxNAKy3htT956PK2s1GwBH5F7Ebc8BDC0FzjsGi8Moo0wbi2ej6vdu2 fTt+EBzl/kz5Q51gnKa5pmZ1Zg/Z+//SEQU4eMGh8veJzxv2QHHJz+QY59PjNHML wr1i4ywfyhZsD9nQ/Tojfa90gX4NjW9NZ5ZJtiJshAmY48sN5L3OnAqBQ2jlFUlI kEtBs/wUXp1nZvrWYVSHGmDRH/duZcv5DdzUIHZrzfWSi24d55xXKbSZrshi2LvU PtLnX4q+km0HmpeX/hjArMU8B8NjyvZWYl2g28wJNTfb2S3SReFGQqufLCssE43D UUrS/bkFV2OYwcDpUcGYkZkgtXWGlvUCAwEAAaOBmDCBlTAdBgNVHQ4EFgQU7RTW 9XEIqmw8DV+LAOTtrX13744wUwYDVR0jBEwwSoAU/za5AJO3639oMJj7AePXbh6S pomhLqQsMCoxKDAmBgNVBAMMH0F6dXJlX0lvVF9IdWJfQ0FfQ2VydF9UZXN0X09u bHmCAhAAMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3 DQEBCwUAA4ICAQB75xeVzuwW8eNBJ+Ztn5PNJbi+OhIGpvWX+aigzI2JQd72BAUR gv8CAeNbib0JYlB/LVZ0TMJRbyJV8BMG+Nen5bl+wehPq1ce0J5QT7cILHL7rpKZ GKbyI3EOuhFG/xc592N72FTz7DYIzYgG7tj/WuidbC7pCNNZfyNerbOkn1It4iae eDwzgrNFPusj0tHFFtIH1nJKxBdjDVlLy6cFHpZlTw5S3qMMg9npgwXCZZvmSYZE DKWFxikl9MIXkvTIHiNxF/C5Ia1WiJeAjTnVwdkAMKrU3s1QKyOlhiTz1rM3I+AQ 89mGKEbVJrE8j1ylbJECCiiE4EnIexTATd6Yw8Fmhl9CGDk7zU+jNX0b2E7G8/eC H2VKoyZWB28q0tfeW607vwephCMDJ2eBQ/UBuBsLON7fakGLRIhfrrQlyFYdEmMV b7ML85mBClVtl41dsmln4RevIJxxOxi71oaJEs0acOrPpZfY+0yw6CvhSVEFAlfD oNl963R/tTSB33stV4wsC+Bfxzq1djlFhFXTlfhPrGPEN9ndLB79rlGOlbtBxR9x 6uoct6k/JBUvmSyQLSOvw5V6fTVd9UPC8KJBsTEXqjcTLLsVPjk+AoIWOKfiOQLQ yIdKYUcLFgRaphS3sOn5ImVmRys++tcuUaW97s+fdOluxr/T25WLovB3tw== -----END CERTIFICATE----- 2 s:CN = Azure_IoT_Hub_Intermediate_Cert_Test_Only i:CN = Azure_IoT_Hub_CA_Cert_Test_Only -----BEGIN CERTIFICATE----- MIIFhTCCA22gAwIBAgICEAAwDQYJKoZIhvcNAQELBQAwKjEoMCYGA1UEAwwfQXp1 cmVfSW9UX0h1Yl9DQV9DZXJ0X1Rlc3RfT25seTAeFw0yMTA2MTgxNDIwMzJaFw0y MTA3MTgxNDIwMzJaMDQxMjAwBgNVBAMMKUF6dXJlX0lvVF9IdWJfSW50ZXJtZWRp YXRlX0NlcnRfVGVzdF9Pbmx5MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKC AgEAqYklnKwrr8GMlwtAdyr025B7wo/g5j4fjethQUloOkCdsAB48cI2sOekAeMF ApszuAc3l/BVV4/9JeFlS/mEokTTQzi9KxZst57Oq3njFORaKdePTwRrERuag7CZ RAVQEJTEmJFnvVNLBxAl61eOV5j7/rRp761pes8tQmaMaArg3mFhdHNATbrErsNE F3VoFf0WftEjn1P9p04L9tENimmfw+uozKDoloEC1l7jGbVfHirk5m0UGbDtdsMU 707bVsf9fqQ7d9CMxbGwdo+WDjSRFE0yAYdQG9KQGELkOMP41gLR6BxJdVXR/6P4 OqTIcNPL4j55aOGJJ4dORA3umDymP1ZPnpejQOu81cqNqPClESQMQUVD8mCvu1Mf tqOiSJmnrGpXQnDJnFsyqMMY3/dQveD/dxH/zDMHxqdot5UIxvU2F4/0gR0kD40S 5k619S2T+FwpL6ip/RjWsRYDsrX0FMrkFBkMSuYGI9TuOYLar65apgNqXmCX7kdL F3a9OndTuuSQMXzSM6q+6vOjufEUEhUmraxuJ/ZCZND3lWQuFohQrFsgyjINCU1+ lQAr0ye/FwsRDP/O8hHrZp58KjQOzDJPC6Nh8uAS+bRaU5S1Xf1fRLv6hfte/Jyf QTVYVDaTm+N21dz35o0J7UBSO/6B/RMNAlrAfWiNnbbRtOUCAwEAAaOBqjCBpzAd BgNVHQ4EFgQU/za5AJO3639oMJj7AePXbh6SpokwZQYDVR0jBF4wXIAUdPm37OCg qMgW6LmO7siQjZiTfR2hLqQsMCoxKDAmBgNVBAMMH0F6dXJlX0lvVF9IdWJfQ0Ff Q2VydF9UZXN0X09ubHmCFEqXe9Y3h7ib1H3Yufh3gcTI0eu9MA8GA1UdEwEB/wQF MAMBAf8wDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4ICAQBdFGTGOHd5 e+SKU9MXfqM060CbQ/YHxkdf2NbQZdsRQHryWK3e2XSX/lMCHQE2NYj7X6oipiSX t9EadthGh34ik+6Yh0JnwsrNCUqT2feVD0I9qYLexW8WgfT2VqJTTSsysl2SqXL1 PwlrAGxFZlKwQO206l7DJg3YdLczqp9AQtOTO/u67GYq9TsMyYo60msqxtrWSXtk NLHqBITthuxR4r28me8C+jmnluWrgzv6zb2jcOuXBxlkR/NHISZ5y80hprifHXdM W9j8vipbZlAbBA34GRhdX2ePH8dq2oZkObryWQPeGeKDXziIHFKHre7irLp3VhyC Qjwxlu6l2irCMxH6S0YKEJS30VGxeFOlh3tkNsjAFnMwEyAasxQ35nvAhzWBCpMc bzs64LZUeA9pYQdkeD5kQy9Qom6Drsu1u96fkLWnLp6lr69fFd2W5kcM39EMBu0A 6ofnmLfw1df9emgT6VqavSBCMQ33b3n59or6KWJUJ+oNUwi9MR1CD36g6PgNZKMo qa+6AGzu19nUu0KLXbiMfKUeqqi7k2BLpJuTUQDaNm4+pa5xnrjzgqdBmxpew6F9 0jJCtbbPFNYiw+nR2NWy9yQLmOS3Q/ITrzYwNraICzrJitkwLYwZ8gThEQxxuTjR 7NP5JfEEoHXxeH0w/FsqQZKtEiuvaNgwHQ== -----END CERTIFICATE----- --- Server certificate subject=CN = xx-yy-ubuntu1804 issuer=CN = IoTEdgeGatewayCA.ca --- No client certificate CA names sent Client Certificate Types: RSA sign, DSA sign, ECDSA sign Requested Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:ECDSA+SHA1:RSA+SHA224:RSA+SHA1:DSA+SHA224:DSA+SHA1:DSA+SHA256:DSA+SHA384:DSA+SHA512 Shared Requested Signature Algorithms: ECDSA+SHA256:ECDSA+SHA384:ECDSA+SHA512:Ed25519:Ed448:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA-PSS+SHA256:RSA-PSS+SHA384:RSA-PSS+SHA512:RSA+SHA256:RSA+SHA384:RSA+SHA512:ECDSA+SHA224:RSA+SHA224:DSA+SHA224:DSA+SHA256:DSA+SHA384:DSA+SHA512 Peer signing digest: SHA256 Peer signature type: RSA-PSS Server Temp Key: X25519, 253 bits --- SSL handshake has read 4511 bytes and written 388 bytes Verification error: unable to get local issuer certificate --- New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384 Server public key is 2048 bit Secure Renegotiation IS supported Compression: NONE Expansion: NONE No ALPN negotiated SSL-Session: Protocol : TLSv1.2 Cipher : ECDHE-RSA-AES256-GCM-SHA384 Session-ID: 0271D0B60FC15AEE7F2E4D624692E8AB47E1F91B140C3B5019BF8C88955E734E Session-ID-ctx: Master-Key: 4DD8AEA0099CB12BA376B82D3D8454C1957A7AA20FF751426E81F1BCE29CFA3898294FB69F6A66D8A80A3287EA5A2972 PSK identity: None PSK identity hint: None SRP username: None TLS session ticket lifetime hint: 7200 (seconds) TLS session ticket: 0000 - 4f ce 3c c5 69 bb 8a 50-e5 89 50 aa 12 ca ed dc O.<.i..P..P..... 0010 - 1d ad 84 9d 96 04 72 09-76 a2 d5 78 dc 69 50 2e ......r.v..x.iP. 0020 - e9 7a ed 4c f2 74 37 76-2a de 75 18 73 15 14 04 .z.L.t7v*.u.s... 0030 - ba 35 1e ba b6 92 0b 46-a8 0e 96 13 b8 a4 4f c9 .5.....F......O. 0040 - 58 cd 2e d3 8b f9 ce 40-a6 08 a8 b5 ce f0 5e 83 X......@......^. 0050 - bc e6 8f e2 a0 fa 6a 7e-05 17 cd 75 a9 f7 7b fd ......j~...u..{. 0060 - 0d fd 6b 86 85 de 61 28-3a d4 9a 45 bc 79 a6 81 ..k...a(:..E.y.. 0070 - 80 79 f2 6c 4d 14 90 92-76 0a c6 ca 5e 65 45 f0 .y.lM...v...^eE. 0080 - 03 3f ba 7a eb c5 18 20-2e 91 89 cb 91 0e 9b 45 .?.z... .......E 0090 - b9 2d 17 53 f2 e9 ce 66-ae 03 17 00 42 b4 97 65 .-.S...f....B..e Start Time: 1624862835 Timeout : 7200 (sec) Verify return code: 20 (unable to get local issuer certificate) Extended master secret: yes ```

In both cases I can connect to the edgeHub using amqp port.

dmolokanov commented 3 years ago

When I'm trying to reproduce the issue I've got a different Exception in edgehub logs which is expected and not blocking edgehub from accepting messages from modules/leaf devices.

<4> 2021-07-01 19:42:39.576 +00:00 [WRN] - Error creating cloud connection for client demo-investigate/SimulatedTemperatureSensor
System.TimeoutException: The operation timed out.
 ---> Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry.
 ---> System.TimeoutException: The operation has timed out.

did you modify hostname in the config.toml file maybe?

if you exec to the SimulatedTemperatureSensor and lookup ENV do you see a hostname is the same as in the config. hostname of the device?

$ sudo docker exec -it SimulatedTemperatureSensor sh
> export
export IOTEDGE_DEVICEID='demo-investigate'
export IOTEDGE_GATEWAYHOSTNAME='ubuntu-demo'
export IOTEDGE_IOTHUBHOSTNAME='demo-iothub.azure-devices.net'
shaeussler commented 3 years ago

Hi @dmolokanov

here are the details you are asking for: sudo docker exec -it SimulatedTemperatureSensor sh export export IOTEDGE_DEVICEID='di-10' export IOTEDGE_GATEWAYHOSTNAME='xx-yy-ubuntu1804' export IOTEDGE_IOTHUBHOSTNAME='XXXXXXXXXXXXXXXX.azure-devices.net'

sudo cat /etc/aziot/config.toml hostname = "xx-yy-ubuntu1804"

hostname xx-yy-ubuntu1804

I can give you access to the VM via ssh?

shaeussler commented 3 years ago

At the moment I can't reproduce the bug. Please close the issue.

github-actions[bot] commented 3 years ago

This issue is being marked as stale because it has been open for 30 days with no activity.

micahl commented 3 years ago

Closing

bender-joe commented 3 years ago

I also experienced this same issue - reproduceable on iotedge version 1.2.0. Upgrading to iotedge v1.2.4 resolved the issue in my scenario.