Module cannot launch on yocto Hardknott Image

wangxuefan123 commented 3 years ago

Expected Behavior

After edgeAgent launch, edgeHub and SimulatedTemperatureSensor should start shortly after.

Current Behavior

edgeAgent seems to run normally, but edgeHub and SimulatedTemperatureSensor failed to start.

Steps to Reproduce

Provide a detailed set of steps to reproduce the bug. 1.Yocto Image compiled with meta-iotedge-1.1.3 and meta-rust-1.47 , meta-virtualization (Hardknott)

Start the iotedge on a custom board

Context (Environment)

Output of `iotedge check`

Click here

``` Configuration checks -------------------- �√ config.yaml is well-formed - OK �√ config.yaml has well-formed connection string - OK �√ container engine is installed and functional - OK �× config.yaml has correct hostname - Error config.yaml has hostname imx6qsabresd-sha001 but device reports hostname imx6qsabresd. Hostname in config.yaml must either be identical to the device hostname or be a fully-qualified domain name that has the device hostname as the first comp. �√ config.yaml has correct URIs for daemon mgmt endpoint - OK �√ latest security daemon - OK �‼ host time is close to real time - Warning Could not query NTP server �√ container time is close to host time - OK �‼ DNS server - Warning Container engine is not configured with DNS server setting, which may impact connectivity to IoT Hub. Please see https://aka.ms/iotedge-prod-checklist-dns for best practices. You can ignore this warning if you are setting DNS server per module in the Edge deployment. �‼ production readiness: certificates - Warning The Edge device is using self-signed automatically-generated development certificates. They will expire in 89 days (at 2021-09-30 00:00:22 UTC) causing module-to-module and downstream device communication to fail on an active deployment. After the certs have expired, restarting the IoT Edge daemon will trigger it to generate new development certs. Please consider using production certificates instead. See https://aka.ms/iotedge-prod-checklist-certs for best practices. �‼ production readiness: container engine - Warning Device is not using a production-supported container engine (moby-engine). Please see https://aka.ms/iotedge-prod-checklist-moby for details. �‼ production readiness: logs policy - Warning Container engine is not configured to rotate module logs which may cause it run out of disk space. Please see https://aka.ms/iotedge-prod-checklist-logs for best practices. You can ignore this warning if you are setting log policy per module in the Edge deployment. �‼ production readiness: Edge Agent's storage directory is persisted on the host filesystem - Warning The edgeAgent module is not configured to persist its /tmp/edgeAgent directory on the host filesystem. Data might be lost if the module is deleted or updated. Please see https://aka.ms/iotedge-storage-host for best practices. �× production readiness: Edge Hub's storage directory is persisted on the host filesystem - Error Could not check current state of edgeHub container Connectivity checks ------------------- �× host can connect to and perform TLS handshake with IoT Hub AMQP port - Error Could not connect to IOThubEdgeScale.azure-devices.net:5671 �√ host can connect to and perform TLS handshake with IoT Hub HTTPS / WebSockets port - OK �× host can connect to and perform TLS handshake with IoT Hub MQTT port - Error Could not connect to IOThubEdgeScale.azure-devices.net:8883 �× container on the default network can connect to IoT Hub AMQP port - Error Container on the default network could not connect to IOThubEdgeScale.azure-devices.net:5671 �× container on the default network can connect to IoT Hub HTTPS / WebSockets port - Error Container on the default network could not connect to IOThubEdgeScale.azure-devices.net:443 �× container on the default network can connect to IoT Hub MQTT port - Error Container on the default network could not connect to IOThubEdgeScale.azure-devices.net:8883 �× container on the IoT Edge module network can connect to IoT Hub AMQP port - Error Container on the azure-iot-edge network could not connect to IOThubEdgeScale.azure-devices.net:5671 �√ container on the IoT Edge module network can connect to IoT Hub HTTPS / WebSockets port - OK �× container on the IoT Edge module network can connect to IoT Hub MQTT port - Error Container on the azure-iot-edge network could not connect to IOThubEdgeScale.azure-devices.net:8883 ```

Device Information

Host OS: Yocto 5.10.35 Hardknott:
Architecture: armv7l
Container OS: docker-ce

Runtime Versions

aziot-edged [run iotedge version]: 1.1.3
Edge Agent [image tag (e.g. 1.0.0)]: 1.1.3
Edge Hub [image tag (e.g. 1.0.0)]: 1.1.3
Docker/Moby : v19.03.15-ce, build 99e3ed8919

Logs

iotedge list

``` edgeAgent running Up 29 minutes mcr.microsoft.com/azureiotedge-agent:1.1 ```

edge-agent logs

``` 2021-07-02 13:26:13 +00:00 Starting Edge Agent 2021-07-02 13:26:13 +00:00 Changing ownership of management socket: /var/run/iotedge/mgmt.sock 2021-07-02 13:26:15.151 +00:00 Edge Agent Main() <6> 2021-07-02 13:26:17.348 +00:00 [INF] - Initializing Edge Agent. <6> 2021-07-02 13:26:18.851 +00:00 [INF] - Version - 1.1.3.42832849 (68e71a5384b14956775241557877497274d0ce7e) <6> 2021-07-02 13:26:18.855 +00:00 [INF] - �█�█�█�█�█�╗ �█�█�█�█�█�█�█�╗�█�█�╗ �█�█�╗�█�█�█�█�█�█�╗ �█�█�█�█�█�█�█�╗ �█�█�╔�═�═�█�█�╗�╚�═�═�█�█�█�╔�╝�█�█�║ �█�█�║�█�█�╔�═�═�█�█�╗�█�█�╔�═�═�═�═�╝ �█�█�█�█�█�█�█�║ �█�█�█�╔�╝ �█�█�║ �█�█�║�█�█�█�█�█�█�╔�╝�█�█�█�█�█�╗ �█�█�╔�═�═�█�█�║ �█�█�█�╔�╝ �█�█�║ �█�█�║�█�█�╔�═�═�█�█�╗�█�█�╔�═�═�╝ �█�█�║ �█�█�║�█�█�█�█�█�█�█�╗�╚�█�█�█�█�█�█�╔�╝�█�█�║ �█�█�║�█�█�█�█�█�█�█�╗ �╚�═�╝ �╚�═�╝�╚�═�═�═�═�═�═�╝ �╚�═�═�═�═�═�╝ �╚�═�╝ �╚�═�╝�╚�═�═�═�═�═�═�╝ �█�█�╗ �█�█�█�█�█�█�╗ �█�█�█�█�█�█�█�█�╗ �█�█�█�█�█�█�█�╗�█�█�█�█�█�█�╗ �█�█�█�█�█�█�╗ �█�█�█�█�█�█�█�╗ �█�█�║�█�█�╔�═�═�═�█�█�╗�╚�═�═�█�█�╔�═�═�╝ �█�█�╔�═�═�═�═�╝�█�█�╔�═�═�█�█�╗�█�█�╔�═�═�═�═�╝ �█�█�╔�═�═� ╝ �█�█�║�█�█�║ �█�█�║ �█�█�║ �█�█�█�█�█�╗ �█�█�║ �█�█�║�█�█�║ �█�█�█�╗�█�█�█�█�█�╗ �█�█�║�█�█�║ �█�█�║ �█�█�║ �█�█�╔�═�═�╝ �█�█�║ �█�█�║�█�█�║ �█�█�║�█�█�╔�═�═�╝ �█�█�║�╚�█�█�█�█�█�█�╔�╝ �█�█�║ �█�█�█�█�█�█�█�╗�█�█�█�█�█�█�╔�╝�╚�█�█�█�█�█�█�╔�╝�█�█�█�█�█�█�█�╗ �╚�═�╝ �╚�═�═�═�═�═�╝ �╚�═�╝ �╚�═�═�═�═�═�═�╝�╚�═�═�═�═�═�╝ �╚�═�═�═�═�═�╝ �╚�═�═�═�═�═�═�╝ <6> 2021-07-02 13:26:19.155 +00:00 [INF] - Experimental features configuration: {"Enabled":false,"DisableCloudSubscriptions":false} <6> 2021-07-02 13:26:20.123 +00:00 [INF] - Installing certificates [CN=Test Edge Owner CA:09/30/2021 00:00:22] to Root <6> 2021-07-02 13:26:21.030 +00:00 [INF] - Starting metrics listener on Host: *, Port: 9600, Suffix: /metrics <6> 2021-07-02 13:26:22.127 +00:00 [INF] - Updating performance metrics every 05m:00s <6> 2021-07-02 13:26:22.155 +00:00 [INF] - Started operation Get system resources <6> 2021-07-02 13:26:22.165 +00:00 [INF] - Collecting metadata metrics <6> 2021-07-02 13:26:22.557 +00:00 [INF] - Set metadata metrics: 1.1.3.42832849 (68e71a5384b14956775241557877497274d0ce7e), {"Enabled":false,"DisableCloudSubse <6> 2021-07-02 13:26:22.716 +00:00 [INF] - Started operation Checkpoint Availability <6> 2021-07-02 13:26:22.768 +00:00 [INF] - Started operation refresh twin config <6> 2021-07-02 13:26:22.880 +00:00 [INF] - Edge agent attempting to connect to IoT Hub via Amqp_Tcp_Only... <6> 2021-07-02 13:26:25.156 +00:00 [INF] - Created persistent store at /tmp/edgeAgent <6> 2021-07-02 13:26:25.425 +00:00 [INF] - Started operation Metrics Scrape <6> 2021-07-02 13:26:25.425 +00:00 [INF] - Started operation Metrics Upload Scraping frequency: 01:00:00 Upload Frequency: 1.00:00:00 <6> 2021-07-02 13:26:26.883 +00:00 [INF] - Registering request handler UploadModuleLogs <6> 2021-07-02 13:26:26.884 +00:00 [INF] - Registering request handler GetModuleLogs <6> 2021-07-02 13:26:26.885 +00:00 [INF] - Registering request handler UploadSupportBundle <6> 2021-07-02 13:26:26.886 +00:00 [INF] - Registering request handler RestartModule <4> 2021-07-02 13:26:32.254 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <6> 2021-07-02 13:26:32.270 +00:00 [INF] - Edge agent config backup file does not exist - /tmp/edgeAgent/backup.json <6> 2021-07-02 13:26:36.835 +00:00 [INF] - Termination requested, initiating shutdown. <6> 2021-07-02 13:26:36.840 +00:00 [INF] - Main thread terminated <6> 2021-07-02 13:26:36.849 +00:00 [INF] - Initiating shutdown cleanup. <6> 2021-07-02 13:26:36.857 +00:00 [INF] - Stopping all modules... <6> 2021-07-02 13:26:36.885 +00:00 [INF] - Waiting for cleanup to finish <6> 2021-07-02 13:26:36.991 +00:00 [INF] - Plan execution started for deployment -1 <6> 2021-07-02 13:26:37.004 +00:00 [INF] - Plan execution ended for deployment -1 <6> 2021-07-02 13:26:37.005 +00:00 [INF] - Completed stopping all modules. <6> 2021-07-02 13:26:37.009 +00:00 [INF] - Completed shutdown cleanup. <6> 2021-07-02 13:26:37.017 +00:00 [INF] - Done with cleanup. Shutting down. Session terminated, terminating shell... ...terminated. 2021-07-02 13:26:39 +00:00 Starting Edge Agent 2021-07-02 13:26:39 +00:00 Changing ownership of storage folder: /tmp/edgeAgent to 1000 2021-07-02 13:26:39 +00:00 Changing ownership of management socket: /var/run/iotedge/mgmt.sock 2021-07-02 13:26:40.338 +00:00 Edge Agent Main() <6> 2021-07-02 13:26:41.433 +00:00 [INF] - Initializing Edge Agent. <6> 2021-07-02 13:26:42.375 +00:00 [INF] - Version - 1.1.3.42832849 (68e71a5384b14956775241557877497274d0ce7e) <6> 2021-07-02 13:26:42.378 +00:00 [INF] - �█�█�█�█�█�╗ �█�█�█�█�█�█�█�╗�█�█�╗ �█�█�╗�█�█�█�█�█�█�╗ �█�█�█�█�█�█�█�╗ �█�█�╔�═�═�█�█�╗�╚�═�═�█�█�█�╔�╝�█�█�║ �█�█�║�█�█�╔�═�═�█�█�╗�█�█�╔�═�═�═�═�╝ �█�█�█�█�█�█�█�║ �█�█�█�╔�╝ �█�█�║ �█�█�║�█�█�█�█�█�█�╔�╝�█�█�█�█�█�╗ �█�█�╔�═�═�█�█�║ �█�█�█�╔�╝ �█�█�║ �█�█�║�█�█�╔�═�═�█�█�╗�█�█�╔�═�═�╝ �█�█�║ �█�█�║�█�█�█�█�█�█�█�╗�╚�█�█�█�█�█�█�╔�╝�█�█�║ �█�█�║�█�█�█�█�█�█�█�╗ �╚�═�╝ �╚�═�╝�╚�═�═�═�═�═�═�╝ �╚�═�═�═�═�═�╝ �╚�═�╝ �╚�═�╝�╚�═�═�═�═�═�═�╝ �█�█�╗ �█�█�█�█�█�█�╗ �█�█�█�█�█�█�█�█�╗ �█�█�█�█�█�█�█�╗�█�█�█�█�█�█�╗ �█�█�█�█�█�█�╗ �█�█�█�█�█�█�█�╗ �█�█�║�█�█�╔�═�═�═�█�█�╗�╚�═�═�█�█�╔�═�═�╝ �█�█�╔�═�═�═�═�╝�█�█�╔�═�═�█�█�╗�█�█�╔�═�═�═�═�╝ �█�█�╔�═�═� ╝ �█�█�║�█�█�║ �█�█�║ �█�█�║ �█�█�█�█�█�╗ �█�█�║ �█�█�║�█�█�║ �█�█�█�╗�█�█�█�█�█�╗ �█�█�║�█�█�║ �█�█�║ �█�█�║ �█�█�╔�═�═�╝ �█�█�║ �█�█�║�█�█�║ �█�█�║�█�█�╔�═�═�╝ �█�█�║�╚�█�█�█�█�█�█�╔�╝ �█�█�║ �█�█�█�█�█�█�█�╗�█�█�█�█�█�█�╔�╝�╚�█�█�█�█�█�█�╔�╝�█�█�█�█�█�█�█�╗ �╚�═�╝ �╚�═�═�═�═�═�╝ �╚�═�╝ �╚�═�═�═�═�═�═�╝�╚�═�═�═�═�═�╝ �╚�═�═�═�═�═�╝ �╚�═�═�═�═�═�═�╝ <6> 2021-07-02 13:26:42.628 +00:00 [INF] - Experimental features configuration: {"Enabled":false,"DisableCloudSubscriptions":false} <6> 2021-07-02 13:26:43.230 +00:00 [INF] - Installing certificates [CN=Test Edge Owner CA:09/30/2021 00:00:22] to Root <6> 2021-07-02 13:26:43.968 +00:00 [INF] - Starting metrics listener on Host: *, Port: 9600, Suffix: /metrics <6> 2021-07-02 13:26:44.813 +00:00 [INF] - Updating performance metrics every 05m:00s <6> 2021-07-02 13:26:44.840 +00:00 [INF] - Started operation Get system resources <6> 2021-07-02 13:26:44.849 +00:00 [INF] - Collecting metadata metrics <6> 2021-07-02 13:26:45.227 +00:00 [INF] - Set metadata metrics: 1.1.3.42832849 (68e71a5384b14956775241557877497274d0ce7e), {"Enabled":false,"DisableCloudSubse <6> 2021-07-02 13:26:45.323 +00:00 [INF] - Started operation Checkpoint Availability <6> 2021-07-02 13:26:45.348 +00:00 [INF] - Started operation refresh twin config <6> 2021-07-02 13:26:45.437 +00:00 [INF] - Edge agent attempting to connect to IoT Hub via Amqp_Tcp_Only... <6> 2021-07-02 13:26:47.169 +00:00 [INF] - Created persistent store at /tmp/edgeAgent <6> 2021-07-02 13:26:47.384 +00:00 [INF] - Started operation Metrics Scrape <6> 2021-07-02 13:26:47.385 +00:00 [INF] - Started operation Metrics Upload Scraping frequency: 01:00:00 Upload Frequency: 1.00:00:00 <6> 2021-07-02 13:26:48.682 +00:00 [INF] - Registering request handler UploadModuleLogs <6> 2021-07-02 13:26:48.684 +00:00 [INF] - Registering request handler GetModuleLogs <6> 2021-07-02 13:26:48.685 +00:00 [INF] - Registering request handler UploadSupportBundle <6> 2021-07-02 13:26:48.685 +00:00 [INF] - Registering request handler RestartModule <4> 2021-07-02 13:26:54.045 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <6> 2021-07-02 13:26:54.061 +00:00 [INF] - Edge agent config backup file does not exist - /tmp/edgeAgent/backup.json <4> 2021-07-02 13:27:04.105 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-07-02 13:27:14.139 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <6> 2021-07-02 13:27:16.644 +00:00 [INF] - Edge agent attempting to connect to IoT Hub via Amqp_WebSocket_Only... <4> 2021-07-02 13:27:24.160 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-07-02 13:27:34.185 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-07-02 13:27:44.217 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-07-02 13:27:47.511 +00:00 [WRN] - Error creating a device-to-cloud connection System.AggregateException: One or more errors occurred. (Transient network error occurred, please retry.) (Transient network error occurred, please retry.) ---> Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.InitializeAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpIoTConnector.OpenConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.OpenSessionAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass23_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass33_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.CreateAndOpenSdkModuleClient(UpstreamProtocol upstreamProtocol, ConnectionStatusChangesHa7 at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.<>c__DisplayClass19_1.<b__2>d.MoveNext() in /home/vsts/work/1/s/ed3 --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Edge.Util.Fallback.ExecuteAsync[T](Func`1[] options) --- End of inner exception stack trace --- ---> (Inner Exception #1) Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Net.WebSockets.WebSocketException (0x80004005): Unable to connect to the remote server ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.WebSockets.WebSocketHandle.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken, ClientWebSocketOptions options) at System.Net.WebSockets.WebSocketHandle.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken, ClientWebSocketOptions options) at System.Net.WebSockets.ClientWebSocket.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.CreateClientWebSocketAsync(Uri websocketUri, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.CreateClientWebSocketTransportAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.InitializeAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpIoTConnector.OpenConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.OpenSessionAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass23_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass33_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.CreateAndOpenSdkModuleClient(UpstreamProtocol upstreamProtocol, ConnectionStatusChangesHa7 at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.<>c__DisplayClass19_1.<b__3>d.MoveNext() in /home/vsts/work/1/s/ed9 --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Edge.Util.Fallback.ExecuteAsync[T](Func`1[] options)<--- <6> 2021-07-02 13:27:47.713 +00:00 [INF] - Edge agent attempting to connect to IoT Hub via Amqp_Tcp_Only... <4> 2021-07-02 13:27:54.237 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-07-02 13:28:04.252 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-07-02 13:28:14.258 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <6> 2021-07-02 13:28:18.231 +00:00 [INF] - Edge agent attempting to connect to IoT Hub via Amqp_WebSocket_Only... <4> 2021-07-02 13:28:24.281 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-07-02 13:28:34.286 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-07-02 13:28:44.300 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-07-02 13:28:48.902 +00:00 [WRN] - Error creating a device-to-cloud connection System.AggregateException: One or more errors occurred. (Transient network error occurred, please retry.) (Transient network error occurred, please retry.) ---> Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.InitializeAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpIoTConnector.OpenConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.OpenSessionAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass23_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass33_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.CreateAndOpenSdkModuleClient(UpstreamProtocol upstreamProtocol, ConnectionStatusChangesHa7 at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.<>c__DisplayClass19_1.<b__2>d.MoveNext() in /home/vsts/work/1/s/ed3 --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Edge.Util.Fallback.ExecuteAsync[T](Func`1[] options) --- End of inner exception stack trace --- ---> (Inner Exception #1) Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Net.WebSockets.WebSocketException (0x80004005): Unable to connect to the remote server ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.WebSockets.WebSocketHandle.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken, ClientWebSocketOptions options) at System.Net.WebSockets.WebSocketHandle.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken, ClientWebSocketOptions options) at System.Net.WebSockets.ClientWebSocket.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.CreateClientWebSocketAsync(Uri websocketUri, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.CreateClientWebSocketTransportAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.InitializeAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpIoTConnector.OpenConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.OpenSessionAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass23_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass33_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.CreateAndOpenSdkModuleClient(UpstreamProtocol upstreamProtocol, ConnectionStatusChangesHa7 at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.<>c__DisplayClass19_1.<b__3>d.MoveNext() in /home/vsts/work/1/s/ed9 --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Edge.Util.Fallback.ExecuteAsync[T](Func`1[] options)<--- <4> 2021-07-02 13:28:54.314 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <6> 2021-07-02 13:28:55.146 +00:00 [INF] - Edge agent attempting to connect to IoT Hub via Amqp_Tcp_Only... <4> 2021-07-02 13:29:04.314 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-07-02 13:29:14.319 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-07-02 13:29:24.328 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <6> 2021-07-02 13:29:25.665 +00:00 [INF] - Edge agent attempting to connect to IoT Hub via Amqp_WebSocket_Only... <4> 2021-07-02 13:29:34.342 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-07-02 13:29:44.362 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-07-02 13:29:54.370 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-07-02 13:29:56.280 +00:00 [WRN] - Error creating a device-to-cloud connection System.AggregateException: One or more errors occurred. (Transient network error occurred, please retry.) (Transient network error occurred, please retry.) ---> Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.InitializeAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpIoTConnector.OpenConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.OpenSessionAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass23_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass33_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.CreateAndOpenSdkModuleClient(UpstreamProtocol upstreamProtocol, ConnectionStatusChangesHa7 at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.<>c__DisplayClass19_1.<b__2>d.MoveNext() in /home/vsts/work/1/s/ed3 --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Edge.Util.Fallback.ExecuteAsync[T](Func`1[] options) --- End of inner exception stack trace --- ---> (Inner Exception #1) Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Net.WebSockets.WebSocketException (1806782968): Unable to connect to the remote server ---> System.Net.Http.HttpRequestException: Resource tempore ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.WebSockets.WebSocketHandle.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken, ClientWebSocketOptions options) at System.Net.WebSockets.WebSocketHandle.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken, ClientWebSocketOptions options) at System.Net.WebSockets.ClientWebSocket.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.CreateClientWebSocketAsync(Uri websocketUri, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.CreateClientWebSocketTransportAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.InitializeAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpIoTConnector.OpenConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.OpenSessionAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass23_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass33_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.CreateAndOpenSdkModuleClient(UpstreamProtocol upstreamProtocol, ConnectionStatusChangesHa7 at Microsoft.Azure.Devices.Edge.Agent.IoTHub.ModuleClientProvider.<>c__DisplayClass19_1.<b__3>d.MoveNext() in /home/vsts/work/1/s/ed9 --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Edge.Util.Fallback.ExecuteAsync[T](Func`1[] options)<--- <4> 2021-07-02 13:30:04.377 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <6> 2021-07-02 13:30:08.199 +00:00 [INF] - Edge agent attempting to connect to IoT Hub via Amqp_Tcp_Only... <4> 2021-07-02 13:30:14.395 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-07-02 13:30:24.407 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <4> 2021-07-02 13:30:34.431 +00:00 [WRN] - Empty edge agent config was received. Attempting to read config from backup (/tmp/edgeAgent/backup.json) instead <6> 2021-07-02 13:30:38.727 +00:00 [INF] - Edge agent attempting to connect to IoT Hub via Amqp_WebSocket_Only... ```

edge-hub logs

``` Jul 02 13:41:38 imx6qsabresd iotedged[750]: 2021-07-02T13:41:38Z [INFO] - Checking edge runtime status Jul 02 13:41:38 imx6qsabresd iotedged[750]: 2021-07-02T13:41:38Z [INFO] - Edge runtime is running. Jul 02 13:41:39 imx6qsabresd iotedged[750]: 2021-07-02T13:41:39Z [INFO] - [mgmt] - - - [2021-07-02 13:41:39.891073778 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 609 "-" "-" auth_id(-) Jul 02 13:41:49 imx6qsabresd iotedged[750]: 2021-07-02T13:41:49Z [INFO] - [mgmt] - - - [2021-07-02 13:41:49.905713113 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 609 "-" "-" auth_id(-) Jul 02 13:41:59 imx6qsabresd iotedged[750]: 2021-07-02T13:41:59Z [INFO] - [mgmt] - - - [2021-07-02 13:41:59.903116114 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 609 "-" "-" auth_id(-) Jul 02 13:42:09 imx6qsabresd iotedged[750]: 2021-07-02T13:42:09Z [INFO] - [mgmt] - - - [2021-07-02 13:42:09.926355448 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 609 "-" "-" auth_id(-) Jul 02 13:42:19 imx6qsabresd iotedged[750]: 2021-07-02T13:42:19Z [INFO] - [mgmt] - - - [2021-07-02 13:42:19.932128116 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 609 "-" "-" auth_id(-) Jul 02 13:42:28 imx6qsabresd iotedged[750]: 2021-07-02T13:42:28Z [INFO] - Getting logs for module edgeHub... Jul 02 13:42:28 imx6qsabresd iotedged[750]: 2021-07-02T13:42:28Z [WARN] - Could not get logs for module edgeHub Jul 02 13:42:28 imx6qsabresd iotedged[750]: 2021-07-02T13:42:28Z [WARN] - caused by: Container runtime error - Api(ApiError { code: 404, content: None }) Jul 02 13:42:28 imx6qsabresd iotedged[750]: 2021-07-02T13:42:28Z [INFO] - [mgmt] - - - [2021-07-02 13:42:28.748926451 UTC] "GET /modules/edgeHub/logs?api-version=2020-07-07&follow=false&tail=all×tamps=false&since=1625146948 HTTP/1.1" 500 Internal Server Error 186 "-" "iotedge/0.1.0" auth_id(-) Jul 02 13:42:29 imx6qsabresd iotedged[750]: 2021-07-02T13:42:29Z [INFO] - [mgmt] - - - [2021-07-02 13:42:29.922291451 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 609 "-" "-" auth_id(-) Jul 02 13:42:38 imx6qsabresd iotedged[750]: 2021-07-02T13:42:38Z [INFO] - Checking edge runtime status Jul 02 13:42:38 imx6qsabresd iotedged[750]: 2021-07-02T13:42:38Z [INFO] - Edge runtime is running. Jul 02 13:42:39 imx6qsabresd iotedged[750]: 2021-07-02T13:42:39Z [INFO] - [mgmt] - - - [2021-07-02 13:42:39.924093119 UTC] "GET /modules?api-version=2020-07-07 HTTP/1.1" 200 OK 609 "-" "-" auth_id(-) Jul 02 13:42:40 imx6qsabresd iotedged[750]: 2021-07-02T13:42:40Z [INFO] - Getting logs for module edgeAgent... Jul 02 13:42:40 imx6qsabresd iotedged[750]: 2021-07-02T13:42:40Z [INFO] - Successfully got logs for module edgeAgent Jul 02 13:42:40 imx6qsabresd iotedged[750]: 2021-07-02T13:42:40Z [INFO] - [mgmt] - - - [2021-07-02 13:42:40.313012452 UTC] "GET /modules/edgeAgent/logs?api-version=2020-07-07&follow=false&tail=all×tamps=false&since=1625146960 HTTP/1.1" 200 OK - "-" "iotedge/0.1.0" auth_id(-) ```

Additional Information

Please provide any additional information that may be helpful in understanding the issue. support_bundle.zip

wangxuefan123 commented 3 years ago

It turns out after setting time ,run

hwclock -w

then edgeHub is able to start, but still meet problem with Simulated TemperatureSensor log

iotedge logs SimulatedTemperatureSensor

``` [2021-07-02 16:47:11 +00:00]: Starting Module SimulatedTemperatureSensor Main() started. Initializing simulated temperature sensor to send 500 messages, at an interval of 5 seconds. To change this, set the environment variable MessageCount to the number of messages that should be sent (set it to -1 to send unlimited messages). [Information]: Trying to initialize module client using transport type [Amqp_Tcp_Only]. Unhandled exception. System.AggregateException: One or more errors occurred. (Transient network error occurred, please retry.) ---> Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: Transient network error occurred, please retry. ---> System.Net.Sockets.SocketException (113): No route to host at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTTransport.InitializeAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpIoTConnector.OpenConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.EnsureConnectionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpConnectionHolder.OpenSessionAsync(DeviceIdentity deviceIdentity, TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.EnsureSessionAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.OpenAsync(TimeSpan timeout) at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ProtocolRoutingDelegatingHandler.OpenAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass23_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) --- End of inner exception stack trace --- at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T](Func`1 asyncOperation) at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.<>c__DisplayClass33_0.<b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.EnsureOpenedAsync(CancellationToken cancellationToken) at Microsoft.Azure.Devices.Client.InternalClient.OpenAsync() at SimulatedTemperatureSensor.Program.<>c__DisplayClass18_0.<b__1>d.MoveNext() in /home/vsts/work/1/s/edge-modules/SimulatedTemperatureSensor/src/Program.cs:line 294 --- End of stack trace from previous location where exception was thrown --- at SimulatedTemperatureSensor.Program.CreateModuleClientAsync(TransportType transportType, ITransientErrorDetectionStrategy transientErrorDetectionStrategy, RetryStrategy retryStrategy) in /home/vsts2 at SimulatedTemperatureSensor.Program.MainAsync() in /home/vsts/work/1/s/edge-modules/SimulatedTemperatureSensor/src/Program.cs:line 79 --- End of inner exception stack trace --- at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification) at System.Threading.Tasks.Task`1.get_Result() at SimulatedTemperatureSensor.Program.Main() in /home/vsts/work/1/s/edge-modules/SimulatedTemperatureSensor/src/Program.cs:line 48 ```

vipeller commented 3 years ago

@wangxuefan123 if you wait until you see edgeHub running and then start SimulatedTemperatureSensor (e.g. using the command 'iotedge restart SimulatedTemperatureSensor', you still see the 'no route to host' message?

wangxuefan123 commented 3 years ago

@wangxuefan123 if you wait until you see edgeHub running and then start SimulatedTemperatureSensor (e.g. using the command 'iotedge restart SimulatedTemperatureSensor', you still see the 'no route to host' message?

Yes, I've tried restart SimulatedTemperatureSensor several times but it just keeps repeating the 'no route to host' message.

github-actions[bot] commented 3 years ago

This issue is being marked as stale because it has been open for 30 days with no activity.

Azure / iotedge