search

Azure / iotedge

The IoT Edge OSS project
MIT License
1.47k stars 463 forks source link

Unexpected shutdown, is it a kind of planned upgrade? #7250

Closed ciacco85 closed 7 months ago

ciacco85 commented 8 months ago

I'm closo to release in production and I've exprerienced a restart in my custom module and I've digged inside $edgeagent log. Since I haven't found anything inside the docs, I want to understand if it's something expected or not. I'm running latest version of every tool and module 

<6> 2024-03-26 14:55:36.243 +00:00 [INF] - Successfully completed periodic operation Metrics Scrape
<6> 2024-03-26 15:55:21.915 +00:00 [INF] - Starting compaction of stores
<6> 2024-03-26 15:55:21.915 +00:00 [INF] - Starting compaction of store Metrics
<6> 2024-03-26 15:55:21.916 +00:00 [INF] - Starting compaction of store default
<6> 2024-03-26 15:55:21.916 +00:00 [INF] - Starting compaction of store moduleState
<6> 2024-03-26 15:55:21.916 +00:00 [INF] - Starting compaction of store deploymentConfig
<6> 2024-03-26 15:55:32.180 +00:00 [INF] - Starting periodic operation refresh twin config...
<6> 2024-03-26 15:55:32.236 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 156 and reported properties version 1067.
<6> 2024-03-26 15:55:32.238 +00:00 [INF] - Successfully completed periodic operation refresh twin config
<6> 2024-03-26 15:55:36.246 +00:00 [INF] - Starting periodic operation Metrics Scrape...
<6> 2024-03-26 15:55:36.246 +00:00 [INF] - Scraping Metrics
<6> 2024-03-26 15:55:36.247 +00:00 [INF] - Scraping endpoint http://edgeHub:9600/metrics
<6> 2024-03-26 15:55:36.248 +00:00 [INF] - Scraping endpoint http://edgeAgent:9600/metrics
<6> 2024-03-26 15:55:36.323 +00:00 [INF] - Storing Metrics
<6> 2024-03-26 15:55:36.348 +00:00 [INF] - Scraped and Stored Metrics
<6> 2024-03-26 15:55:36.348 +00:00 [INF] - Successfully completed periodic operation Metrics Scrape
<6> 2024-03-26 16:02:14.156 +00:00 [INF] - Termination requested, initiating shutdown.
<6> 2024-03-26 16:02:14.157 +00:00 [INF] - Main thread terminated
<6> 2024-03-26 16:02:14.168 +00:00 [INF] - Initiating shutdown cleanup.
<6> 2024-03-26 16:02:14.174 +00:00 [INF] - Stopping all modules...
<6> 2024-03-26 16:02:14.220 +00:00 [INF] - Plan execution started for deployment -1
<6> 2024-03-26 16:02:14.227 +00:00 [INF] - Executing command: "Stop module edgeHub"
<6> 2024-03-26 16:02:14.230 +00:00 [INF] - Executing command: "Stop module glema-badger"
<6> 2024-03-26 16:02:14.274 +00:00 [INF] - Waiting for cleanup to finish
<6> 2024-03-26 16:02:14.437 +00:00 [INF] - Updated reported properties
2024-03-26 16:02:27  Starting Edge Agent
2024-03-26 16:02:27  Changing ownership of storage folder: /tmp/edgeAgent to 13622
2024-03-26 16:02:27  Changing ownership of backup folder: /tmp/edgeAgent_backup to 13622
2024-03-26 16:02:27  Changing ownership of management socket: /var/run/iotedge/mgmt.sock
2024-03-26 16:02:27  Completed necessary setup. Starting Edge Agent.
2024-03-26 16:02:27.283 +00:00 Edge Agent Main()
<6> 2024-03-26 16:02:27.945 +00:00 [INF] - Initializing Edge Agent.
<6> 2024-03-26 16:02:28.394 +00:00 [INF] - Version - 1.4.33.89334537 (71f4974213cfab7447117af13a30319ba2c40357)
<6> 2024-03-26 16:02:28.396 +00:00 [INF] - 
        █████╗ ███████╗██╗   ██╗██████╗ ███████╗
       ██╔══██╗╚══███╔╝██║   ██║██╔══██╗██╔════╝
       ███████║  ███╔╝ ██║   ██║██████╔╝█████╗
       ██╔══██║ ███╔╝  ██║   ██║██╔══██╗██╔══╝
       ██║  ██║███████╗╚██████╔╝██║  ██║███████╗
       ╚═╝  ╚═╝╚══════╝ ╚═════╝ ╚═╝  ╚═╝╚══════╝

 ██╗ ██████╗ ████████╗    ███████╗██████╗  ██████╗ ███████╗
 ██║██╔═══██╗╚══██╔══╝    ██╔════╝██╔══██╗██╔════╝ ██╔════╝
 ██║██║   ██║   ██║       █████╗  ██║  ██║██║  ███╗█████╗
 ██║██║   ██║   ██║       ██╔══╝  ██║  ██║██║   ██║██╔══╝
 ██║╚██████╔╝   ██║       ███████╗██████╔╝╚██████╔╝███████╗
 ╚═╝ ╚═════╝    ╚═╝       ╚══════╝╚═════╝  ╚═════╝ ╚══════╝

<6> 2024-03-26 16:02:28.406 +00:00 [INF] - ModuleUpdateMode: NonBlocking
<6> 2024-03-26 16:02:28.585 +00:00 [INF] - Experimental features configuration: {"Enabled":false,"DisableCloudSubscriptions":false}
<6> 2024-03-26 16:02:29.092 +00:00 [INF] - Installing certificates [CN=aziot-edge CA dev-01:06/24/2024 16:02:14] to Root
<6> 2024-03-26 16:02:29.606 +00:00 [INF] - Starting metrics listener on Host: *, Port: 9600, Suffix: metrics
<6> 2024-03-26 16:02:29.653 +00:00 [INF] - Updating performance metrics every 05m:00s
<6> 2024-03-26 16:02:29.668 +00:00 [INF] - Started operation Get system resources
<6> 2024-03-26 16:02:29.674 +00:00 [INF] - Collecting metadata metrics
<6> 2024-03-26 16:02:29.906 +00:00 [INF] - Set metadata metrics: 1.4.33.89334537 (71f4974213cfab7447117af13a30319ba2c40357), {"Enabled":false,"DisableCloudSubscriptions":false}, {"OperatingSystemType":"Linux","Architecture":"arm","Version":"1.4.27","Provisioning":{"Type":"ProvisioningType","DynamicReprovisioning":false,"AlwaysReprovisionOnStartup":false},"ServerVersion":"24.0.9-1","KernelVersion":"#1642 SMP PREEMPT Mon Apr  3 17:24:16 BST 2023","OperatingSystem":"debian","NumCpus":4,"TotalMemory":8191807488,"Virtualized":"no"}, True
<6> 2024-03-26 16:02:29.974 +00:00 [INF] - Started operation Checkpoint Availability
<6> 2024-03-26 16:02:29.984 +00:00 [INF] - Started operation refresh twin config
<6> 2024-03-26 16:02:30.059 +00:00 [INF] - Edge agent attempting to connect to IoT Hub via Amqp_Tcp_Only...
<6> 2024-03-26 16:02:31.171 +00:00 [INF] - Created persistent store at /tmp/edgeAgent
<6> 2024-03-26 16:02:31.467 +00:00 [INF] - Started operation Metrics Scrape
<6> 2024-03-26 16:02:31.467 +00:00 [INF] - Started operation Metrics Upload
Scraping frequency: 01:00:00
Upload Frequency: 1.00:00:00
<6> 2024-03-26 16:02:33.873 +00:00 [INF] - Registering request handler UploadModuleLogs
<6> 2024-03-26 16:02:33.874 +00:00 [INF] - Registering request handler GetModuleLogs
<6> 2024-03-26 16:02:33.874 +00:00 [INF] - Registering request handler UploadSupportBundle
<6> 2024-03-26 16:02:33.875 +00:00 [INF] - Registering request handler RestartModule
<6> 2024-03-26 16:02:33.996 +00:00 [INF] - Edge agent connected to IoT Hub via Amqp_Tcp_Only.
<6> 2024-03-26 16:02:34.674 +00:00 [INF] - Initialized new module client with subscriptions enabled
<6> 2024-03-26 16:02:34.980 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 156 and reported properties version 1068.
<6> 2024-03-26 16:02:37.308 +00:00 [INF] - Plan execution started for deployment 156
<6> 2024-03-26 16:02:37.317 +00:00 [INF] - Executing command: "[Null]"
<6> 2024-03-26 16:02:37.331 +00:00 [INF] - Executing command: "Saving glema-badger to store"
<6> 2024-03-26 16:02:37.424 +00:00 [INF] - Executing command: "Command Group: (\n  [Command Group: (\n  [Prepare module glema-badger]\n  [Command Group: (\n  [Stop module glema-badger]\n  [Update module glema-badger]\n)]\n)]\n  [Start module glema-badger]\n)"
<6> 2024-03-26 16:02:37.432 +00:00 [INF] - Executing command: "Command Group: (\n  [Prepare module glema-badger]\n  [Command Group: (\n  [Stop module glema-badger]\n  [Update module glema-badger]\n)]\n)"
<6> 2024-03-26 16:02:40.795 +00:00 [INF] - Executing command: "Start module glema-badger"
<6> 2024-03-26 16:02:41.511 +00:00 [INF] - Plan execution ended for deployment 156
<6> 2024-03-26 16:02:41.984 +00:00 [INF] - Updated reported properties
<6> 2024-03-26 16:02:47.207 +00:00 [INF] - Updated reported properties
<6> 2024-03-26 17:02:29.984 +00:00 [INF] - Starting periodic operation refresh twin config...
<6> 2024-03-26 17:02:30.044 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 156 and reported properties version 1070.
<6> 2024-03-26 17:02:30.049 +00:00 [INF] - Successfully completed periodic operation refresh twin config
<6> 2024-03-26 17:02:31.467 +00:00 [INF] - Starting periodic operation Metrics Scrape...
<6> 2024-03-26 17:02:31.471 +00:00 [INF] - Scraping Metrics
<6> 2024-03-26 17:02:31.481 +00:00 [INF] - Scraping endpoint http://edgeHub:9600/metrics
<6> 2024-03-26 17:02:31.565 +00:00 [INF] - Scraping endpoint http://edgeAgent:9600/metrics
<6> 2024-03-26 17:02:31.733 +00:00 [INF] - Storing Metrics
<6> 2024-03-26 17:02:31.894 +00:00 [INF] - Scraped and Stored Metrics
<6> 2024-03-26 17:02:31.895 +00:00 [INF] - Successfully completed periodic operation Metrics Scrape
<6> 2024-03-26 17:02:32.603 +00:00 [INF] - Plan execution started for deployment 156
<6> 2024-03-26 17:02:32.604 +00:00 [INF] - Executing command: "[Null]"
<6> 2024-03-26 17:02:32.604 +00:00 [INF] - Executing command: "Stop module glema-badger"
<6> 2024-03-26 17:02:33.256 +00:00 [INF] - Executing command: "Remove module glema-badger"
<6> 2024-03-26 17:02:33.339 +00:00 [INF] - Executing command: "Saving glema-badger to store"
<6> 2024-03-26 17:02:33.340 +00:00 [INF] - Executing command: "Saving glema-badger to store"
<6> 2024-03-26 17:02:33.341 +00:00 [INF] - Executing command: "Command Group: (\n  [Command Group: (\n  [Prepare module glema-badger]\n  [Command Group: (\n  [Stop module glema-badger]\n  [Update module glema-badger]\n)]\n)]\n  [Start module glema-badger]\n)"
<6> 2024-03-26 17:02:33.341 +00:00 [INF] - Executing command: "Command Group: (\n  [Prepare module glema-badger]\n  [Command Group: (\n  [Stop module glema-badger]\n  [Update module glema-badger]\n)]\n)"
<3> 2024-03-26 17:02:34.029 +00:00 [ERR] - Executing command for operation ["update"] failed.
Microsoft.Azure.Devices.Edge.Agent.Edgelet.EdgeletCommunicationException- Message:Error calling stop module glema-badger: runtime operation error: stop module "glema-badger", StatusCode:404, at:   at Microsoft.Azure.Devices.Edge.Agent.Edgelet.Version_2022_08_03.ModuleManagementHttpClient.HandleException(Exception exception, String operation) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Edgelet/version_2022_08_03/ModuleManagementHttpClient.cs:line 232
   at Microsoft.Azure.Devices.Edge.Agent.Edgelet.Versioning.ModuleManagementHttpClientVersioned.Execute[T](Func`1 func, String operation) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Edgelet/versioning/ModuleManagementHttpClientVersioned.cs:line 168
   at Microsoft.Azure.Devices.Edge.Agent.Edgelet.Version_2022_08_03.ModuleManagementHttpClient.StopModuleAsync(String name) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Edgelet/version_2022_08_03/ModuleManagementHttpClient.cs:line 187
   at Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient.<>c__DisplayClass26_0.<<Throttle>b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Edgelet/ModuleManagementHttpClient.cs:line 143
--- End of stack trace from previous location ---
   at Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient.Throttle[T](Func`1 identityOperation) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Edgelet/ModuleManagementHttpClient.cs:line 158
   at Microsoft.Azure.Devices.Edge.Agent.Core.Commands.GroupCommand.ExecuteAsync(CancellationToken token) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/commands/GroupCommand.cs:line 35
   at Microsoft.Azure.Devices.Edge.Agent.Core.Commands.GroupCommand.ExecuteAsync(CancellationToken token) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/commands/GroupCommand.cs:line 35
   at Microsoft.Azure.Devices.Edge.Agent.Core.LoggingCommandFactory.LoggingCommand.ExecuteAsync(CancellationToken token) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/LoggingCommandFactory.cs:line 61
<3> 2024-03-26 17:02:34.075 +00:00 [ERR] - Executing command for operation ["Command Group: (\n  [Command Group: (\n  [Prepare module glema-badger]\n  [Command Group: (\n  [Stop module glema-badger]\n  [Update module glema-badger]\n)]\n)]\n  [Start module glema-badger]\n)"] failed.
Microsoft.Azure.Devices.Edge.Agent.Edgelet.EdgeletCommunicationException- Message:Error calling stop module glema-badger: runtime operation error: stop module "glema-badger", StatusCode:404, at:   at Microsoft.Azure.Devices.Edge.Agent.Edgelet.Version_2022_08_03.ModuleManagementHttpClient.HandleException(Exception exception, String operation) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Edgelet/version_2022_08_03/ModuleManagementHttpClient.cs:line 232
   at Microsoft.Azure.Devices.Edge.Agent.Edgelet.Versioning.ModuleManagementHttpClientVersioned.Execute[T](Func`1 func, String operation) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Edgelet/versioning/ModuleManagementHttpClientVersioned.cs:line 168
   at Microsoft.Azure.Devices.Edge.Agent.Edgelet.Version_2022_08_03.ModuleManagementHttpClient.StopModuleAsync(String name) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Edgelet/version_2022_08_03/ModuleManagementHttpClient.cs:line 187
   at Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient.<>c__DisplayClass26_0.<<Throttle>b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Edgelet/ModuleManagementHttpClient.cs:line 143
--- End of stack trace from previous location ---
   at Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient.Throttle[T](Func`1 identityOperation) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Edgelet/ModuleManagementHttpClient.cs:line 158
   at Microsoft.Azure.Devices.Edge.Agent.Core.Commands.GroupCommand.ExecuteAsync(CancellationToken token) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/commands/GroupCommand.cs:line 35
   at Microsoft.Azure.Devices.Edge.Agent.Core.Commands.GroupCommand.ExecuteAsync(CancellationToken token) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/commands/GroupCommand.cs:line 35
   at Microsoft.Azure.Devices.Edge.Agent.Core.LoggingCommandFactory.LoggingCommand.ExecuteAsync(CancellationToken token) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/LoggingCommandFactory.cs:line 61
   at Microsoft.Azure.Devices.Edge.Agent.Core.Commands.GroupCommand.ExecuteAsync(CancellationToken token) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/commands/GroupCommand.cs:line 35
   at Microsoft.Azure.Devices.Edge.Agent.Core.LoggingCommandFactory.LoggingCommand.ExecuteAsync(CancellationToken token) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/LoggingCommandFactory.cs:line 61
<3> 2024-03-26 17:02:34.084 +00:00 [ERR] - Step failed in deployment 156, continuing execution. Failure when running command Command Group: (
  [Command Group: (
  [Prepare module glema-badger]
  [Command Group: (
  [Stop module glema-badger]
  [Update module glema-badger]
)]
)]
  [Start module glema-badger]
). Will retry in 00s.
<6> 2024-03-26 17:02:34.086 +00:00 [INF] - Plan execution ended for deployment 156
<3> 2024-03-26 17:02:34.088 +00:00 [ERR] - Edge agent plan execution failed.
System.AggregateException: One or more errors occurred. (Error calling stop module glema-badger: runtime operation error: stop module "glema-badger")
 ---> Microsoft.Azure.Devices.Edge.Agent.Edgelet.EdgeletCommunicationException- Message:Error calling stop module glema-badger: runtime operation error: stop module "glema-badger", StatusCode:404, at:   at Microsoft.Azure.Devices.Edge.Agent.Edgelet.Version_2022_08_03.ModuleManagementHttpClient.HandleException(Exception exception, String operation) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Edgelet/version_2022_08_03/ModuleManagementHttpClient.cs:line 232
   at Microsoft.Azure.Devices.Edge.Agent.Edgelet.Versioning.ModuleManagementHttpClientVersioned.Execute[T](Func`1 func, String operation) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Edgelet/versioning/ModuleManagementHttpClientVersioned.cs:line 168
   at Microsoft.Azure.Devices.Edge.Agent.Edgelet.Version_2022_08_03.ModuleManagementHttpClient.StopModuleAsync(String name) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Edgelet/version_2022_08_03/ModuleManagementHttpClient.cs:line 187
   at Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient.<>c__DisplayClass26_0.<<Throttle>b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Edgelet/ModuleManagementHttpClient.cs:line 143
--- End of stack trace from previous location ---
   at Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient.Throttle[T](Func`1 identityOperation) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Edgelet/ModuleManagementHttpClient.cs:line 158
   at Microsoft.Azure.Devices.Edge.Agent.Core.Commands.GroupCommand.ExecuteAsync(CancellationToken token) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/commands/GroupCommand.cs:line 35
   at Microsoft.Azure.Devices.Edge.Agent.Core.Commands.GroupCommand.ExecuteAsync(CancellationToken token) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/commands/GroupCommand.cs:line 35
   at Microsoft.Azure.Devices.Edge.Agent.Core.LoggingCommandFactory.LoggingCommand.ExecuteAsync(CancellationToken token) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/LoggingCommandFactory.cs:line 61
   at Microsoft.Azure.Devices.Edge.Agent.Core.Commands.GroupCommand.ExecuteAsync(CancellationToken token) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/commands/GroupCommand.cs:line 35
   at Microsoft.Azure.Devices.Edge.Agent.Core.LoggingCommandFactory.LoggingCommand.ExecuteAsync(CancellationToken token) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/LoggingCommandFactory.cs:line 61
   at Microsoft.Azure.Devices.Edge.Agent.Core.PlanRunner.OrderedRetryPlanRunner.ExecuteAsync(Int64 deploymentId, Plan plan, CancellationToken token) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/planrunner/OrdererdRetryPlanRunner.cs:line 91
   --- End of inner exception stack trace ---
   at Microsoft.Azure.Devices.Edge.Agent.Core.PlanRunner.OrderedRetryPlanRunner.<>c.<ExecuteAsync>b__7_0(List`1 f) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/planrunner/OrdererdRetryPlanRunner.cs:line 129
   at Microsoft.Azure.Devices.Edge.Agent.Core.PlanRunner.OrderedRetryPlanRunner.ExecuteAsync(Int64 deploymentId, Plan plan, CancellationToken token)
   at Microsoft.Azure.Devices.Edge.Agent.Core.Agent.ReconcileAsync(CancellationToken token) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/Agent.cs:line 208
<4> 2024-03-26 17:02:34.130 +00:00 [WRN] - Reconcile failed because of the an exception
System.AggregateException: One or more errors occurred. (Error calling stop module glema-badger: runtime operation error: stop module "glema-badger")
 ---> Microsoft.Azure.Devices.Edge.Agent.Edgelet.EdgeletCommunicationException- Message:Error calling stop module glema-badger: runtime operation error: stop module "glema-badger", StatusCode:404, at:   at Microsoft.Azure.Devices.Edge.Agent.Edgelet.Version_2022_08_03.ModuleManagementHttpClient.HandleException(Exception exception, String operation) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Edgelet/version_2022_08_03/ModuleManagementHttpClient.cs:line 232
   at Microsoft.Azure.Devices.Edge.Agent.Edgelet.Versioning.ModuleManagementHttpClientVersioned.Execute[T](Func`1 func, String operation) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Edgelet/versioning/ModuleManagementHttpClientVersioned.cs:line 168
   at Microsoft.Azure.Devices.Edge.Agent.Edgelet.Version_2022_08_03.ModuleManagementHttpClient.StopModuleAsync(String name) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Edgelet/version_2022_08_03/ModuleManagementHttpClient.cs:line 187
   at Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient.<>c__DisplayClass26_0.<<Throttle>b__0>d.MoveNext() in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Edgelet/ModuleManagementHttpClient.cs:line 143
--- End of stack trace from previous location ---
   at Microsoft.Azure.Devices.Edge.Agent.Edgelet.ModuleManagementHttpClient.Throttle[T](Func`1 identityOperation) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Edgelet/ModuleManagementHttpClient.cs:line 158
   at Microsoft.Azure.Devices.Edge.Agent.Core.Commands.GroupCommand.ExecuteAsync(CancellationToken token) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/commands/GroupCommand.cs:line 35
   at Microsoft.Azure.Devices.Edge.Agent.Core.Commands.GroupCommand.ExecuteAsync(CancellationToken token) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/commands/GroupCommand.cs:line 35
   at Microsoft.Azure.Devices.Edge.Agent.Core.LoggingCommandFactory.LoggingCommand.ExecuteAsync(CancellationToken token) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/LoggingCommandFactory.cs:line 61
   at Microsoft.Azure.Devices.Edge.Agent.Core.Commands.GroupCommand.ExecuteAsync(CancellationToken token) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/commands/GroupCommand.cs:line 35
   at Microsoft.Azure.Devices.Edge.Agent.Core.LoggingCommandFactory.LoggingCommand.ExecuteAsync(CancellationToken token) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/LoggingCommandFactory.cs:line 61
   at Microsoft.Azure.Devices.Edge.Agent.Core.PlanRunner.OrderedRetryPlanRunner.ExecuteAsync(Int64 deploymentId, Plan plan, CancellationToken token) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/planrunner/OrdererdRetryPlanRunner.cs:line 91
   --- End of inner exception stack trace ---
   at Microsoft.Azure.Devices.Edge.Agent.Core.PlanRunner.OrderedRetryPlanRunner.<>c.<ExecuteAsync>b__7_0(List`1 f) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/planrunner/OrdererdRetryPlanRunner.cs:line 129
   at Microsoft.Azure.Devices.Edge.Agent.Core.PlanRunner.OrderedRetryPlanRunner.ExecuteAsync(Int64 deploymentId, Plan plan, CancellationToken token)
   at Microsoft.Azure.Devices.Edge.Agent.Core.Agent.ReconcileAsync(CancellationToken token) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/Agent.cs:line 208
   at Microsoft.Azure.Devices.Edge.Agent.Core.Agent.ReconcileAsync(CancellationToken token) in /mnt/vss/_work/1/s/edge-agent/src/Microsoft.Azure.Devices.Edge.Agent.Core/Agent.cs:line 208
<6> 2024-03-26 17:02:34.309 +00:00 [INF] - Updated reported properties
<6> 2024-03-26 17:02:39.818 +00:00 [INF] - Plan execution started for deployment 156
<6> 2024-03-26 17:02:39.818 +00:00 [INF] - Executing command: "[Null]"
<6> 2024-03-26 17:02:39.819 +00:00 [INF] - Executing command: "Command Group: (\n  [Command Group: (\n  [Prepare module glema-badger]\n  [Create module glema-badger]\n)]\n  [Start module glema-badger]\n)"
<6> 2024-03-26 17:02:39.819 +00:00 [INF] - Executing command: "Command Group: (\n  [Prepare module glema-badger]\n  [Create module glema-badger]\n)"
<6> 2024-03-26 17:02:41.406 +00:00 [INF] - Executing command: "Start module glema-badger"
<6> 2024-03-26 17:02:42.017 +00:00 [INF] - Plan execution ended for deployment 156
<6> 2024-03-26 17:02:42.372 +00:00 [INF] - Updated reported properties
<6> 2024-03-26 17:02:47.600 +00:00 [INF] - Updated reported properties
<6> 2024-03-26 18:02:30.051 +00:00 [INF] - Starting periodic operation refresh twin config...
<6> 2024-03-26 18:02:30.100 +00:00 [INF] - Obtained Edge agent twin from IoTHub with desired properties version 156 and reported properties version 1073.
<6> 2024-03-26 18:02:30.103 +00:00 [INF] - Successfully completed periodic operation refresh twin config
<6> 2024-03-26 18:02:31.179 +00:00 [INF] - Starting compaction of stores
<6> 2024-03-26 18:02:31.180 +00:00 [INF] - Starting compaction of store deploymentConfig
<6> 2024-03-26 18:02:31.183 +00:00 [INF] - Starting compaction of store default
nyanzebra commented 8 months ago

@ciacco85 at least from the above it appears you probably did some deployment and there was an issue trying to stop glema-badger module. It then looks like a bit later the deployment succeeded. It is a little hard to say exactly what happened, but it could be the case that the module glema-badger was gone and the deployment step to stop it failed, but then a little later it was present and it worked. Do you know what the state of your module was when the deployment occurred?

david-emakenemi commented 8 months ago

@ciacco85 any updates?

ciacco85 commented 8 months ago

Sorry for the delay. Honestly I haven't done any deployment, that's why I've opened the issue and, form the log, it seems that the IoT hub has fired this "restart". It hasn't happend anymore since the issue was opened. I think we can close if you haven't other suggestion why this event has happend

nyanzebra commented 7 months ago

Closing issue, if this reoccurs please reopen :)