search

aws / amazon-ssm-agent

An agent to enable remote management of your EC2 instances, on-premises servers, or virtual machines (VMs).
https://aws.amazon.com/systems-manager/
Apache License 2.0
1.06k stars 322 forks source link

AWS-RunShellScript with "exit 194" hangs and fails #289

Closed dorintt closed 4 years ago

dorintt commented 4 years ago

I have SSM-Agent installed on-prem and trying to reboot instances using Run Command, doing it as described in the docs: https://docs.aws.amazon.com/systems-manager/latest/userguide/send-commands-reboot.html

However, after sending the command, the status is "In Progress" for a very long time. During this time, the instances appear online but they can't be accessed using Session Manager, because it says they are not connected. After about 30 minutes, the instances finally restart, and the command fails:

Thor-Bjorgvinsson commented 4 years ago

Hi Dorintt, thanks for reporting this issue. Could you please provide the following information:

  1. What agent version are you running?
  2. What OS/Arch are you running?
  3. Do you have access to the agent logs? If so, is there anything in there that could help us get to the bottom of this issue?

It seems to me that the document is reaching the agent since the instance finally restarts after 30 minutes. There should be some information about this delay in the agent logs

dorintt commented 4 years ago

The problem is happening with all these setups:

The logs are huge because the agent seems to have stuck into a loop, I tried removing the repetitive parts. It's possible that the instances actually restarted immediately, but did so multiple times and only after 30 min they broke out of the loop so that I could connect back and check.

Basically, I requested the command from aws console at 2020-06-23 14:30:41 and this is what follows on one of the instances:

2020-06-23 14:30:41 INFO [MessagingDeliveryService] [messageID=aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71] Validating SSM parameters
2020-06-23 14:30:41 INFO [MessagingDeliveryService] Sending reply {
  "additionalInfo": {
    "agent": {
      "lang": "en-US",
      "name": "amazon-ssm-agent",
      "os": "",
      "osver": "1",
      "ver": ""
    },
    "dateTime": "2020-06-23T11:30:41.704Z",
    "runId": "",
    "runtimeStatusCounts": null
  },
  "documentStatus": "InProgress",
  "documentTraceOutput": "",
  "runtimeStatus": null
}
2020-06-23 14:30:41 INFO [MessagingDeliveryService] SendReply Response{
  Description: "Reply 10d02b1f-f5c0-428f-b8e7-44a18499f75f was successfully sent.",
  MessageId: "aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71",
  ReplyId: "10d02b1f-f5c0-428f-b8e7-44a18499f75f",
  ReplyStatus: "QUEUED"
}
2020-06-23 14:30:42 INFO parsing args: [/usr/bin/ssm-document-worker 1485d8ae-277b-4fb5-8b42-9c002051d690 mi-0bda32df7d3617d71]
2020-06-23 14:30:41 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] channel: 1485d8ae-277b-4fb5-8b42-9c002051d690 not found, creating a new file channel...
2020-06-23 14:30:42 INFO using channelName 1485d8ae-277b-4fb5-8b42-9c002051d690, instanceID: mi-0bda32df7d3617d71
2020-06-23 14:30:41 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] inter process communication started
2020-06-23 14:30:42 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] document: 1485d8ae-277b-4fb5-8b42-9c002051d690 worker started
2020-06-23 14:30:42 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] channel: 1485d8ae-277b-4fb5-8b42-9c002051d690 found
2020-06-23 14:30:42 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:configurePackage
2020-06-23 14:30:42 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:runDocument
2020-06-23 14:30:42 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:softwareInventory
2020-06-23 14:30:42 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:runPowerShellScript
2020-06-23 14:30:42 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:updateSsmAgent
2020-06-23 14:30:42 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:runDockerAction
2020-06-23 14:30:42 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:refreshAssociation
2020-06-23 14:30:42 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:configureDocker
2020-06-23 14:30:42 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:downloadContent
2020-06-23 14:30:42 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform dependent plugin aws:runShellScript
2020-06-23 14:30:42 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] inter process communication started
2020-06-23 14:30:42 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] received plugin config message
2020-06-23 14:30:42 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] {"DocumentInformation":{"DocumentID":"1485d8ae-277b-4fb5-8b42-9c002051d690","CommandID":"1485d8ae-277b-4fb5-8b42-9c002051d690","AssociationID":"","InstanceID":"mi-0bda32df7d3617d71","MessageID":"aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71","RunID":"2020-06-23T11-30-41.643Z","CreatedDate":"2020-06-23T11:30:41.574Z","DocumentName":"AWS-RunShellScript","DocumentVersion":"","DocumentStatus":"InProgress","RunCount":0,"ProcInfo":{"Pid":19733,"StartTime":"2020-06-23T11:30:41.947745831Z"},"ClientId":"","RunAsUser":""},"DocumentType":"SendCommand","SchemaVersion":"1.2","InstancePluginsInformation":[{"Configuration":{"Settings":null,"Properties":[{"id":"0.aws:runShellScript","runCommand":["exit 194","echo \"rebooted\""],"timeoutSeconds":"3600","workingDirectory":""}],"OutputS3KeyPrefix":"1485d8ae-277b-4fb5-8b42-9c002051d690/mi-0bda32df7d3617d71/awsrunShellScript","OutputS3BucketName":"","S3EncryptionEnabled":false,"CloudWatchLogGroup":"","CloudWatchEncryptionEnabled":false,"OrchestrationDirectory":"/var/lib/amazon/ssm/mi-0bda32df7d3617d71/document/orchestration/1485d8ae-277b-4fb5-8b42-9c002051d690/awsrunShellScript","MessageId":"aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71","BookKeepingFileName":"1485d8ae-277b-4fb5-8b42-9c002051d690","PluginName":"aws:runShellScript","PluginID":"aws:runShellScript","DefaultWorkingDirectory":"","Preconditions":null,"IsPreconditionEnabled":false,"CurrentAssociations":null,"SessionId":"","ClientId":"","KmsKeyId":"","RunAsEnabled":false,"RunAsUser":""},"Name":"aws:runShellScript","Result":{"pluginID":"","pluginName":"","status":"","code":0,"output":null,"startDateTime":"0001-01-01T00:00:00Z","endDateTime":"0001-01-01T00:00:00Z","outputS3BucketName":"","outputS3KeyPrefix":"","stepName":"","error":"","standardOutput":"","standardError":""},"Id":"aws:runShellScript"}],"CancelInformation":{"CancelMessageID":"","CancelCommandID":"","Payload":"","DebugInfo":""},"IOConfig":{"OrchestrationDirectory":"/var/lib/amazon/ssm/mi-0bda32df7d3617d71/document/orchestration/1485d8ae-277b-4fb5-8b42-9c002051d690","OutputS3BucketName":"","OutputS3KeyPrefix":"1485d8ae-277b-4fb5-8b42-9c002051d690/mi-0bda32df7d3617d71","CloudWatchConfig":{"LogGroupName":"/aws/ssm/AWS-RunShellScript","LogStreamPrefix":"1485d8ae-277b-4fb5-8b42-9c002051d690/mi-0bda32df7d3617d71","LogGroupEncryptionEnabled":false}}}
2020-06-23 14:30:42 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] Running plugin aws:runShellScript
2020-06-23 14:30:42 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] [pluginName=aws:runShellScript] aws:runShellScript started with configuration {<nil> map[id:0.aws:runShellScript runCommand:[exit 194 echo "rebooted"] timeoutSeconds:3600 workingDirectory:] 1485d8ae-277b-4fb5-8b42-9c002051d690/mi-0bda32df7d3617d71/awsrunShellScript  false  false /var/lib/amazon/ssm/mi-0bda32df7d3617d71/document/orchestration/1485d8ae-277b-4fb5-8b42-9c002051d690/awsrunShellScript aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71 1485d8ae-277b-4fb5-8b42-9c002051d690 aws:runShellScript aws:runShellScript  map[] false []    false }
2020-06-23 14:30:42 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] [pluginName=aws:runShellScript] The execution of command returned Exit Status: 194
2020-06-23 14:30:48 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] Sending plugin aws:runShellScript completion message
2020-06-23 14:30:48 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] document execution complete
2020-06-23 14:30:48 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] requested terminate messaging worker, destroying the channel
2020-06-23 14:30:48 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] sending document complete response...
2020-06-23 14:30:48 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] stopping ipc worker...
2020-06-23 14:30:48 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] requested shutdown, prepare to stop messaging
2020-06-23 14:30:48 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] channel /var/lib/amazon/ssm/mi-0bda32df7d3617d71/channels/1485d8ae-277b-4fb5-8b42-9c002051d690 requested close
2020-06-23 14:30:48 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] ipc channel closed, stop messaging worker
2020-06-23 14:30:48 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] document worker closed
2020-06-23 14:30:48 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] channel /var/lib/amazon/ssm/mi-0bda32df7d3617d71/channels/1485d8ae-277b-4fb5-8b42-9c002051d690 requested close
2020-06-23 14:30:48 INFO [MessagingDeliveryService] [EngineProcessor] sending reply for plugin update: aws:runShellScript
2020-06-23 14:30:48 INFO [MessagingDeliveryService] [EngineProcessor] sending document: 1485d8ae-277b-4fb5-8b42-9c002051d690 complete response
2020-06-23 14:30:48 INFO [MessagingDeliveryService] received plugin: aws:runShellScript result from Processor
2020-06-23 14:30:48 INFO [MessagingDeliveryService] Sending reply {
  "additionalInfo": {
    "agent": {
      "lang": "en-US",
      "name": "amazon-ssm-agent",
      "os": "",
      "osver": "1",
      "ver": ""
    },
    "dateTime": "2020-06-23T11:30:48.595Z",
    "runId": "",
    "runtimeStatusCounts": {
      "SuccessAndReboot": 1
    }
  },
  "documentStatus": "InProgress",
  "documentTraceOutput": "",
  "runtimeStatus": {
    "aws:runShellScript": {
      "status": "SuccessAndReboot",
      "code": 194,
      "name": "aws:runShellScript",
      "output": "",
      "startDateTime": "2020-06-23T11:30:42.012Z",
      "endDateTime": "2020-06-23T11:30:48.593Z",
      "outputS3BucketName": "",
      "outputS3KeyPrefix": "",
      "stepName": "",
      "standardOutput": "",
      "standardError": ""
    }
  }
}
2020-06-23 14:30:48 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] Executer closed
2020-06-23 14:30:48 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] channel /var/lib/amazon/ssm/mi-0bda32df7d3617d71/channels/1485d8ae-277b-4fb5-8b42-9c002051d690 closed
2020-06-23 14:30:48 INFO [MessagingDeliveryService] SendReply Response{
  Description: "Reply c2665be2-e444-41cc-9ab8-2511bb762acc was successfully sent.",
  MessageId: "aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71",
  ReplyId: "c2665be2-e444-41cc-9ab8-2511bb762acc",
  ReplyStatus: "QUEUED"
}
2020-06-23 14:30:48 INFO [MessagingDeliveryService] command: aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71 complete
2020-06-23 14:30:48 INFO [MessagingDeliveryService] [EngineProcessor] document aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71 requested reboot, need to resume
2020-06-23 14:30:48 INFO [MessagingDeliveryService] [EngineProcessor] successfully requested a reboot
2020-06-23 14:30:48 INFO [MessagingDeliveryService] Sending reply {
  "additionalInfo": {
    "agent": {
      "lang": "en-US",
      "name": "amazon-ssm-agent",
      "os": "",
      "osver": "1",
      "ver": ""
    },
    "dateTime": "2020-06-23T11:30:48.910Z",
    "runId": "",
    "runtimeStatusCounts": {
      "SuccessAndReboot": 1
    }
  },
  "documentStatus": "SuccessAndReboot",
  "documentTraceOutput": "",
  "runtimeStatus": {
    "aws:runShellScript": {
      "status": "SuccessAndReboot",
      "code": 194,
      "name": "aws:runShellScript",
      "output": "",
      "startDateTime": "2020-06-23T11:30:42.012Z",
      "endDateTime": "2020-06-23T11:30:48.593Z",
      "outputS3BucketName": "",
      "outputS3KeyPrefix": "",
      "stepName": "",
      "standardOutput": "",
      "standardError": ""
    }
  }
}
2020-06-23 14:30:48 INFO [instanceID=mi-0bda32df7d3617d71] A plugin has requested a reboot.
2020-06-23 14:30:48 INFO [instanceID=mi-0bda32df7d3617d71] Processing reboot request...
2020-06-23 14:30:48 INFO [instanceID=mi-0bda32df7d3617d71] core manager stop requested. Stop type: SoftStop
2020-06-23 14:30:48 INFO [instanceID=mi-0bda32df7d3617d71] Rebooting the machine in +1 Minutes..
2020-06-23 14:30:48 INFO [HealthCheck] stopping update instance health job.
2020-06-23 14:30:48 INFO [MessageGatewayService] Stopping MessageGatewayService.
2020-06-23 14:30:48 INFO [MessageGatewayService] Closing controlchannel with channel Id mi-0bda32df7d3617d71
2020-06-23 14:30:48 INFO [MessageGatewayService] Closing websocket channel connection to: wss://ssmmessages.eu-central-1.amazonaws.com/v1/control-channel/mi-0bda32df7d3617d71?role=subscribe&stream=input
2020-06-23 14:30:48 INFO [MessageGatewayService] Ending the channel listening routine since the channel is closed
2020-06-23 14:30:48 INFO [instanceID=mi-0bda32df7d3617d71] shutdown output: 

2020-06-23 14:30:48 INFO [MessageGatewayService] Successfully closed websocket connection to: 52.119.189.13:443
2020-06-23 14:30:49 INFO [MessagingDeliveryService] SendReply Response{
  Description: "Reply aee150cf-3321-4127-9603-399f78a492c1 was successfully sent.",
  MessageId: "aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71",
  ReplyId: "aee150cf-3321-4127-9603-399f78a492c1",
  ReplyStatus: "QUEUED"
}
2020-06-23 14:31:48 INFO Got signal:terminated value:0x55cb26428080
2020-06-23 14:31:48 INFO Stopping agent
2020-06-23 14:31:48 INFO [instanceID=mi-0bda32df7d3617d71] core manager stop requested. Stop type: HardStop
2020-06-23 14:31:48 INFO [HealthCheck] stopping update instance health job.
2020-06-23 14:31:48 INFO [MessageGatewayService] Stopping MessageGatewayService.
2020-06-23 14:31:48 INFO [MessageGatewayService] Closing controlchannel with channel Id mi-0bda32df7d3617d71
2020-06-23 14:31:48 INFO [MessageGatewayService] Closing websocket channel connection to: wss://ssmmessages.eu-central-1.amazonaws.com/v1/control-channel/mi-0bda32df7d3617d71?role=subscribe&stream=input
2020-06-23 14:31:53 INFO Bye.
2020-06-23 14:32:29 INFO Create new startup processor
2020-06-23 14:32:29 INFO [LongRunningPluginsManager] registered plugins: {}
2020-06-23 14:32:29 INFO Initializing bookkeeping folders
2020-06-23 14:32:29 INFO removing the completed state files
2020-06-23 14:32:29 INFO Initializing bookkeeping folders for long running plugins
2020-06-23 14:32:29 INFO Initializing replies folder for MDS reply requests that couldn't reach the service
2020-06-23 14:32:29 INFO Initializing healthcheck folders for long running plugins
2020-06-23 14:32:29 INFO Initializing locations for inventory plugin
2020-06-23 14:32:29 INFO Initializing default location for custom inventory
2020-06-23 14:32:29 INFO Initializing default location for file inventory
2020-06-23 14:32:29 INFO Initializing default location for role inventory
2020-06-23 14:32:29 INFO Init the cloudwatchlogs publisher
2020-06-23 14:32:29 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:configurePackage
2020-06-23 14:32:29 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:downloadContent
2020-06-23 14:32:29 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:runDocument
2020-06-23 14:32:29 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:softwareInventory
2020-06-23 14:32:29 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:runPowerShellScript
2020-06-23 14:32:29 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:refreshAssociation
2020-06-23 14:32:29 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:updateSsmAgent
2020-06-23 14:32:29 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:configureDocker
2020-06-23 14:32:29 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:runDockerAction
2020-06-23 14:32:29 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform dependent plugin aws:runShellScript
2020-06-23 14:32:29 INFO Starting Agent: amazon-ssm-agent - v2.3.722.0
2020-06-23 14:32:29 INFO OS: linux, Arch: amd64
2020-06-23 14:32:29 INFO parsing args: [/usr/bin/ssm-document-worker 1485d8ae-277b-4fb5-8b42-9c002051d690 mi-0bda32df7d3617d71]
2020-06-23 14:32:29 INFO [MessageGatewayService] Starting session document processing engine...
2020-06-23 14:32:29 INFO using channelName 1485d8ae-277b-4fb5-8b42-9c002051d690, instanceID: mi-0bda32df7d3617d71
2020-06-23 14:32:29 INFO [MessagingDeliveryService] Starting document processing engine...
2020-06-23 14:32:29 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] document: 1485d8ae-277b-4fb5-8b42-9c002051d690 worker started
2020-06-23 14:32:29 INFO [MessagingDeliveryService] [EngineProcessor] Starting
2020-06-23 14:32:29 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] channel: 1485d8ae-277b-4fb5-8b42-9c002051d690 found
2020-06-23 14:32:29 INFO [HealthCheck] HealthCheck reporting agent health.
2020-06-23 14:32:29 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:softwareInventory
2020-06-23 14:32:29 INFO [MessagingDeliveryService] [EngineProcessor] Initial processing
2020-06-23 14:32:29 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:runPowerShellScript
2020-06-23 14:32:29 INFO [OfflineService] Starting document processing engine...
2020-06-23 14:32:29 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:configurePackage
2020-06-23 14:32:29 INFO [OfflineService] [EngineProcessor] Starting
2020-06-23 14:32:29 INFO [OfflineService] [EngineProcessor] Initial processing
2020-06-23 14:32:29 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:downloadContent
2020-06-23 14:32:29 INFO [LongRunningPluginsManager] starting long running plugin manager
2020-06-23 14:32:29 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:runDocument
2020-06-23 14:32:29 INFO [MessageGatewayService] [EngineProcessor] Starting
2020-06-23 14:32:29 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:updateSsmAgent
2020-06-23 14:32:29 INFO [MessageGatewayService] SSM Agent is trying to setup control channel for Session Manager module.
2020-06-23 14:32:29 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:configureDocker
2020-06-23 14:32:29 INFO [MessageGatewayService] Setting up websocket for controlchannel for instance: mi-0bda32df7d3617d71, requestId: f791149b-b6ae-4837-a825-e581621a55c9
2020-06-23 14:32:29 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:runDockerAction
2020-06-23 14:32:29 INFO [MessageGatewayService] listening reply.
2020-06-23 14:32:29 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:refreshAssociation
2020-06-23 14:32:29 INFO [LongRunningPluginsManager] there aren't any long running plugin to execute
2020-06-23 14:32:29 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform dependent plugin aws:runShellScript
2020-06-23 14:32:29 INFO [LongRunningPluginsManager] There are no long running plugins currently getting executed - skipping their healthcheck
2020-06-23 14:32:29 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] inter process communication started
2020-06-23 14:32:29 INFO [OfflineService] [EngineProcessor] Found in-progress document - 1485d8ae-277b-4fb5-8b42-9c002051d690
2020-06-23 14:32:29 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] received plugin config message
2020-06-23 14:32:29 INFO [MessagingDeliveryService] [EngineProcessor] Found in-progress document - 1485d8ae-277b-4fb5-8b42-9c002051d690
2020-06-23 14:32:29 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] {"DocumentInformation":{"DocumentID":"1485d8ae-277b-4fb5-8b42-9c002051d690","CommandID":"1485d8ae-277b-4fb5-8b42-9c002051d690","AssociationID":"","InstanceID":"mi-0bda32df7d3617d71","MessageID":"aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71","RunID":"2020-06-23T11-30-41.643Z","CreatedDate":"2020-06-23T11:30:41.574Z","DocumentName":"AWS-RunShellScript","DocumentVersion":"","DocumentStatus":"SuccessAndReboot","RunCount":1,"ProcInfo":{"Pid":1215,"StartTime":"2020-06-23T11:32:29.267137321Z"},"ClientId":"","RunAsUser":""},"DocumentType":"SendCommand","SchemaVersion":"1.2","InstancePluginsInformation":[{"Configuration":{"Settings":null,"Properties":[{"id":"0.aws:runShellScript","runCommand":["exit 194","echo \"rebooted\""],"timeoutSeconds":"3600","workingDirectory":""}],"OutputS3KeyPrefix":"1485d8ae-277b-4fb5-8b42-9c002051d690/mi-0bda32df7d3617d71/awsrunShellScript","OutputS3BucketName":"","S3EncryptionEnabled":false,"CloudWatchLogGroup":"","CloudWatchEncryptionEnabled":false,"OrchestrationDirectory":"/var/lib/amazon/ssm/mi-0bda32df7d3617d71/document/orchestration/1485d8ae-277b-4fb5-8b42-9c002051d690/awsrunShellScript","MessageId":"aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71","BookKeepingFileName":"1485d8ae-277b-4fb5-8b42-9c002051d690","PluginName":"aws:runShellScript","PluginID":"aws:runShellScript","DefaultWorkingDirectory":"","Preconditions":null,"IsPreconditionEnabled":false,"CurrentAssociations":null,"SessionId":"","ClientId":"","KmsKeyId":"","RunAsEnabled":false,"RunAsUser":""},"Name":"aws:runShellScript","Result":{"pluginID":"aws:runShellScript","pluginName":"aws:runShellScript","status":"SuccessAndReboot","code":194,"output":"","startDateTime":"2020-06-23T14:30:42.012557706+03:00","endDateTime":"2020-06-23T14:30:48.593507149+03:00","outputS3BucketName":"","outputS3KeyPrefix":"","stepName":"","error":"","standardOutput":"","standardError":""},"Id":"aws:runShellScript"}],"CancelInformation":{"CancelMessageID":"","CancelCommandID":"","Payload":"","DebugInfo":""},"IOConfig":{"OrchestrationDirectory":"/var/lib/amazon/ssm/mi-0bda32df7d3617d71/document/orchestration/1485d8ae-277b-4fb5-8b42-9c002051d690","OutputS3BucketName":"","OutputS3KeyPrefix":"1485d8ae-277b-4fb5-8b42-9c002051d690/mi-0bda32df7d3617d71","CloudWatchConfig":{"LogGroupName":"/aws/ssm/AWS-RunShellScript","LogStreamPrefix":"1485d8ae-277b-4fb5-8b42-9c002051d690/mi-0bda32df7d3617d71","LogGroupEncryptionEnabled":false}}}
2020-06-23 14:32:29 INFO [MessagingDeliveryService] [EngineProcessor] Processing in-progress document 1485d8ae-277b-4fb5-8b42-9c002051d690
2020-06-23 14:32:29 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] Running plugin aws:runShellScript
2020-06-23 14:32:29 INFO [MessagingDeliveryService] Starting message polling
2020-06-23 14:32:29 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] [pluginName=aws:runShellScript] aws:runShellScript started with configuration {<nil> map[id:0.aws:runShellScript runCommand:[exit 194 echo "rebooted"] timeoutSeconds:3600 workingDirectory:] 1485d8ae-277b-4fb5-8b42-9c002051d690/mi-0bda32df7d3617d71/awsrunShellScript  false  false /var/lib/amazon/ssm/mi-0bda32df7d3617d71/document/orchestration/1485d8ae-277b-4fb5-8b42-9c002051d690/awsrunShellScript aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71 1485d8ae-277b-4fb5-8b42-9c002051d690 aws:runShellScript aws:runShellScript  map[] false []    false }
2020-06-23 14:32:29 INFO [MessagingDeliveryService] Starting send replies to MDS
2020-06-23 14:32:29 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] [pluginName=aws:runShellScript] The execution of command returned Exit Status: 194
2020-06-23 14:32:29 INFO [instanceID=mi-0bda32df7d3617d71] Starting association polling
2020-06-23 14:32:29 INFO [MessagingDeliveryService] [Association] [EngineProcessor] Starting
2020-06-23 14:32:29 INFO [MessagingDeliveryService] [Association] Launching response handler
2020-06-23 14:32:29 INFO [MessagingDeliveryService] [Association] [EngineProcessor] Initial processing
2020-06-23 14:32:29 INFO [MessagingDeliveryService] [Association] [EngineProcessor] Found in-progress document - 1485d8ae-277b-4fb5-8b42-9c002051d690
2020-06-23 14:32:29 INFO [OfflineService] Starting message polling
2020-06-23 14:32:29 INFO [OfflineService] Starting send replies to MDS
2020-06-23 14:32:29 INFO [MessagingDeliveryService] [Association] Initializing association scheduling service
2020-06-23 14:32:29 INFO [MessagingDeliveryService] [Association] Association scheduling service initialized
2020-06-23 14:32:29 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] channel: 1485d8ae-277b-4fb5-8b42-9c002051d690 not found, creating a new file channel...
2020-06-23 14:32:29 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] inter process communication started
2020-06-23 14:32:29 INFO [MessageGatewayService] Opening websocket connection to: wss://ssmmessages.eu-central-1.amazonaws.com/v1/control-channel/mi-0bda32df7d3617d71?role=subscribe&stream=input
2020-06-23 14:32:29 INFO [MessageGatewayService] Successfully opened websocket connection to: wss://ssmmessages.eu-central-1.amazonaws.com/v1/control-channel/mi-0bda32df7d3617d71?role=subscribe&stream=input
2020-06-23 14:32:29 INFO [MessageGatewayService] Starting receiving message from control channel
2020-06-23 14:32:29 INFO [MessageGatewayService] [EngineProcessor] Initial processing
2020-06-23 14:32:29 INFO [MessageGatewayService] [EngineProcessor] Found in-progress document - 1485d8ae-277b-4fb5-8b42-9c002051d690
2020-06-23 14:32:30 INFO [MessagingDeliveryService] [Association] No associations on boot. Requerying for associations after 30 seconds.
2020-06-23 14:32:35 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] Sending plugin aws:runShellScript completion message
2020-06-23 14:32:35 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] document execution complete
2020-06-23 14:32:35 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] sending document complete response...
2020-06-23 14:32:35 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] stopping ipc worker...
2020-06-23 14:32:35 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] requested shutdown, prepare to stop messaging
2020-06-23 14:32:35 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] channel /var/lib/amazon/ssm/mi-0bda32df7d3617d71/channels/1485d8ae-277b-4fb5-8b42-9c002051d690 requested close
2020-06-23 14:32:35 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] ipc channel closed, stop messaging worker
2020-06-23 14:32:35 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] document worker closed
2020-06-23 14:32:35 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] requested terminate messaging worker, destroying the channel
2020-06-23 14:32:35 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] channel /var/lib/amazon/ssm/mi-0bda32df7d3617d71/channels/1485d8ae-277b-4fb5-8b42-9c002051d690 requested close
2020-06-23 14:32:35 INFO [MessagingDeliveryService] [EngineProcessor] sending reply for plugin update: aws:runShellScript
2020-06-23 14:32:35 INFO [MessagingDeliveryService] [EngineProcessor] sending document: 1485d8ae-277b-4fb5-8b42-9c002051d690 complete response
2020-06-23 14:32:35 INFO [MessagingDeliveryService] received plugin: aws:runShellScript result from Processor
2020-06-23 14:32:35 INFO [MessagingDeliveryService] Sending reply {
  "additionalInfo": {
    "agent": {
      "lang": "en-US",
      "name": "amazon-ssm-agent",
      "os": "",
      "osver": "1",
      "ver": ""
    },
    "dateTime": "2020-06-23T11:32:35.921Z",
    "runId": "",
    "runtimeStatusCounts": {
      "SuccessAndReboot": 1
    }
  },
  "documentStatus": "InProgress",
  "documentTraceOutput": "",
  "runtimeStatus": {
    "aws:runShellScript": {
      "status": "SuccessAndReboot",
      "code": 194,
      "name": "aws:runShellScript",
      "output": "",
      "startDateTime": "2020-06-23T11:30:42.012Z",
      "endDateTime": "2020-06-23T11:32:35.918Z",
      "outputS3BucketName": "",
      "outputS3KeyPrefix": "",
      "stepName": "",
      "standardOutput": "",
      "standardError": ""
    }
  }
}
2020-06-23 14:32:35 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] Executer closed
2020-06-23 14:32:35 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] channel /var/lib/amazon/ssm/mi-0bda32df7d3617d71/channels/1485d8ae-277b-4fb5-8b42-9c002051d690 closed
2020-06-23 14:32:36 INFO [MessagingDeliveryService] SendReply Response{
  Description: "Reply a3bffcf1-1cfa-422d-8dac-3b7a3f39a2c6 was successfully sent.",
  MessageId: "aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71",
  ReplyId: "a3bffcf1-1cfa-422d-8dac-3b7a3f39a2c6",
  ReplyStatus: "QUEUED"
}
2020-06-23 14:32:36 INFO [MessagingDeliveryService] command: aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71 complete
2020-06-23 14:32:36 INFO [MessagingDeliveryService] [EngineProcessor] document aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71 requested reboot, need to resume
2020-06-23 14:32:36 INFO [MessagingDeliveryService] [EngineProcessor] successfully requested a reboot
2020-06-23 14:32:36 INFO [instanceID=mi-0bda32df7d3617d71] A plugin has requested a reboot.
2020-06-23 14:32:36 INFO [instanceID=mi-0bda32df7d3617d71] Processing reboot request...
2020-06-23 14:32:36 INFO [MessagingDeliveryService] Sending reply {
  "additionalInfo": {
    "agent": {
      "lang": "en-US",
      "name": "amazon-ssm-agent",
      "os": "",
      "osver": "1",
      "ver": ""
    },
    "dateTime": "2020-06-23T11:32:36.335Z",
    "runId": "",
    "runtimeStatusCounts": {
      "SuccessAndReboot": 1
    }
  },
  "documentStatus": "SuccessAndReboot",
  "documentTraceOutput": "",
  "runtimeStatus": {
    "aws:runShellScript": {
      "status": "SuccessAndReboot",
      "code": 194,
      "name": "aws:runShellScript",
      "output": "",
      "startDateTime": "2020-06-23T11:30:42.012Z",
      "endDateTime": "2020-06-23T11:32:35.918Z",
      "outputS3BucketName": "",
      "outputS3KeyPrefix": "",
      "stepName": "",
      "standardOutput": "",
      "standardError": ""
    }
  }
}
2020-06-23 14:32:36 INFO [instanceID=mi-0bda32df7d3617d71] core manager stop requested. Stop type: SoftStop
2020-06-23 14:32:36 INFO [instanceID=mi-0bda32df7d3617d71] Rebooting the machine in +1 Minutes..
2020-06-23 14:32:36 INFO [MessageGatewayService] Stopping MessageGatewayService.
2020-06-23 14:32:36 INFO [MessageGatewayService] Closing controlchannel with channel Id mi-0bda32df7d3617d71
2020-06-23 14:32:36 INFO [MessageGatewayService] Closing websocket channel connection to: wss://ssmmessages.eu-central-1.amazonaws.com/v1/control-channel/mi-0bda32df7d3617d71?role=subscribe&stream=input
2020-06-23 14:32:36 INFO [MessageGatewayService] Ending the channel listening routine since the channel is closed
2020-06-23 14:32:36 INFO [MessageGatewayService] Successfully closed websocket connection to: 52.119.189.13:443
2020-06-23 14:32:36 INFO [instanceID=mi-0bda32df7d3617d71] shutdown output: 

2020-06-23 14:32:36 INFO [MessagingDeliveryService] SendReply Response{
  Description: "Reply 27fc637a-fe9f-430e-9949-1dc2f82e3dc6 was successfully sent.",
  MessageId: "aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71",
  ReplyId: "27fc637a-fe9f-430e-9949-1dc2f82e3dc6",
  ReplyStatus: "QUEUED"
}
2020-06-23 14:33:00 INFO [MessagingDeliveryService] [Association] Schedule manager refreshed with 0 associations, 0 new associations associated
2020-06-23 14:33:11 INFO [HealthCheck] HealthCheck reporting agent health.
2020-06-23 14:33:36 INFO Got signal:terminated value:0x560bbffea080
2020-06-23 14:33:36 INFO Stopping agent
2020-06-23 14:33:36 INFO [instanceID=mi-0bda32df7d3617d71] core manager stop requested. Stop type: HardStop
2020-06-23 14:33:36 INFO [MessageGatewayService] Stopping MessageGatewayService.
2020-06-23 14:33:36 INFO [MessageGatewayService] Closing controlchannel with channel Id mi-0bda32df7d3617d71
2020-06-23 14:33:36 INFO [MessageGatewayService] Closing websocket channel connection to: wss://ssmmessages.eu-central-1.amazonaws.com/v1/control-channel/mi-0bda32df7d3617d71?role=subscribe&stream=input
2020-06-23 14:33:36 INFO [HealthCheck] stopping update instance health job.
2020-06-23 14:33:41 INFO Bye.
2020-06-23 14:34:41 INFO Create new startup processor
2020-06-23 14:34:41 INFO [LongRunningPluginsManager] registered plugins: {}
2020-06-23 14:34:41 INFO Initializing bookkeeping folders
2020-06-23 14:34:41 INFO removing the completed state files
2020-06-23 14:34:41 INFO Initializing bookkeeping folders for long running plugins
2020-06-23 14:34:41 INFO Initializing replies folder for MDS reply requests that couldn't reach the service
2020-06-23 14:34:41 INFO Initializing healthcheck folders for long running plugins
2020-06-23 14:34:41 INFO Initializing locations for inventory plugin
2020-06-23 14:34:41 INFO Initializing default location for custom inventory
2020-06-23 14:34:41 INFO Initializing default location for file inventory
2020-06-23 14:34:41 INFO Initializing default location for role inventory
2020-06-23 14:34:41 INFO Init the cloudwatchlogs publisher
2020-06-23 14:34:41 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:downloadContent
2020-06-23 14:34:41 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:softwareInventory
2020-06-23 14:34:41 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:runPowerShellScript
2020-06-23 14:34:41 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:updateSsmAgent
2020-06-23 14:34:41 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:configureDocker
2020-06-23 14:34:41 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:runDockerAction
2020-06-23 14:34:41 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:refreshAssociation
2020-06-23 14:34:41 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:configurePackage
2020-06-23 14:34:41 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:runDocument
2020-06-23 14:34:41 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform dependent plugin aws:runShellScript
2020-06-23 14:34:41 INFO Starting Agent: amazon-ssm-agent - v2.3.722.0
2020-06-23 14:34:41 INFO OS: linux, Arch: amd64
2020-06-23 14:34:41 INFO parsing args: [/usr/bin/ssm-document-worker 1485d8ae-277b-4fb5-8b42-9c002051d690 mi-0bda32df7d3617d71]
2020-06-23 14:34:41 INFO [MessagingDeliveryService] Starting document processing engine...
2020-06-23 14:34:41 INFO using channelName 1485d8ae-277b-4fb5-8b42-9c002051d690, instanceID: mi-0bda32df7d3617d71
2020-06-23 14:34:41 INFO [OfflineService] Starting document processing engine...
2020-06-23 14:34:41 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] document: 1485d8ae-277b-4fb5-8b42-9c002051d690 worker started
2020-06-23 14:34:41 INFO [OfflineService] [EngineProcessor] Starting
2020-06-23 14:34:41 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] channel: 1485d8ae-277b-4fb5-8b42-9c002051d690 found
2020-06-23 14:34:41 INFO [OfflineService] [EngineProcessor] Initial processing
2020-06-23 14:34:41 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:softwareInventory
2020-06-23 14:34:41 INFO [MessagingDeliveryService] [EngineProcessor] Starting
2020-06-23 14:34:41 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:configureDocker
2020-06-23 14:34:41 INFO [MessagingDeliveryService] [EngineProcessor] Initial processing
2020-06-23 14:34:41 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:runDockerAction
2020-06-23 14:34:41 INFO [OfflineService] [EngineProcessor] Found in-progress document - 1485d8ae-277b-4fb5-8b42-9c002051d690
2020-06-23 14:34:41 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:runPowerShellScript
2020-06-23 14:34:41 INFO [HealthCheck] HealthCheck reporting agent health.
2020-06-23 14:34:41 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:updateSsmAgent
2020-06-23 14:34:41 INFO [LongRunningPluginsManager] starting long running plugin manager
2020-06-23 14:34:41 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:refreshAssociation
2020-06-23 14:34:41 INFO [MessagingDeliveryService] [EngineProcessor] Found in-progress document - 1485d8ae-277b-4fb5-8b42-9c002051d690
2020-06-23 14:34:41 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:configurePackage
2020-06-23 14:34:41 INFO [MessageGatewayService] Starting session document processing engine...
2020-06-23 14:34:41 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:downloadContent
2020-06-23 14:34:41 INFO [MessageGatewayService] [EngineProcessor] Starting
2020-06-23 14:34:41 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:runDocument
2020-06-23 14:34:41 INFO [MessageGatewayService] SSM Agent is trying to setup control channel for Session Manager module.
2020-06-23 14:34:41 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform dependent plugin aws:runShellScript
2020-06-23 14:34:41 INFO [LongRunningPluginsManager] there aren't any long running plugin to execute
2020-06-23 14:34:41 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] inter process communication started
2020-06-23 14:34:41 INFO [MessageGatewayService] Setting up websocket for controlchannel for instance: mi-0bda32df7d3617d71, requestId: 5bcc657c-35dd-4fed-83b7-32445bc2f3ed
2020-06-23 14:34:41 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] received plugin config message
2020-06-23 14:34:41 INFO [LongRunningPluginsManager] There are no long running plugins currently getting executed - skipping their healthcheck
2020-06-23 14:34:41 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] {"DocumentInformation":{"DocumentID":"1485d8ae-277b-4fb5-8b42-9c002051d690","CommandID":"1485d8ae-277b-4fb5-8b42-9c002051d690","AssociationID":"","InstanceID":"mi-0bda32df7d3617d71","MessageID":"aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71","RunID":"2020-06-23T11-30-41.643Z","CreatedDate":"2020-06-23T11:30:41.574Z","DocumentName":"AWS-RunShellScript","DocumentVersion":"","DocumentStatus":"SuccessAndReboot","RunCount":2,"ProcInfo":{"Pid":1207,"StartTime":"2020-06-23T11:34:41.024508432Z"},"ClientId":"","RunAsUser":""},"DocumentType":"SendCommand","SchemaVersion":"1.2","InstancePluginsInformation":[{"Configuration":{"Settings":null,"Properties":[{"id":"0.aws:runShellScript","runCommand":["exit 194","echo \"rebooted\""],"timeoutSeconds":"3600","workingDirectory":""}],"OutputS3KeyPrefix":"1485d8ae-277b-4fb5-8b42-9c002051d690/mi-0bda32df7d3617d71/awsrunShellScript","OutputS3BucketName":"","S3EncryptionEnabled":false,"CloudWatchLogGroup":"","CloudWatchEncryptionEnabled":false,"OrchestrationDirectory":"/var/lib/amazon/ssm/mi-0bda32df7d3617d71/document/orchestration/1485d8ae-277b-4fb5-8b42-9c002051d690/awsrunShellScript","MessageId":"aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71","BookKeepingFileName":"1485d8ae-277b-4fb5-8b42-9c002051d690","PluginName":"aws:runShellScript","PluginID":"aws:runShellScript","DefaultWorkingDirectory":"","Preconditions":null,"IsPreconditionEnabled":false,"CurrentAssociations":null,"SessionId":"","ClientId":"","KmsKeyId":"","RunAsEnabled":false,"RunAsUser":""},"Name":"aws:runShellScript","Result":{"pluginID":"aws:runShellScript","pluginName":"aws:runShellScript","status":"SuccessAndReboot","code":194,"output":"","startDateTime":"2020-06-23T14:30:42.012557706+03:00","endDateTime":"2020-06-23T14:32:35.918466398+03:00","outputS3BucketName":"","outputS3KeyPrefix":"","stepName":"","error":"","standardOutput":"","standardError":""},"Id":"aws:runShellScript"}],"CancelInformation":{"CancelMessageID":"","CancelCommandID":"","Payload":"","DebugInfo":""},"IOConfig":{"OrchestrationDirectory":"/var/lib/amazon/ssm/mi-0bda32df7d3617d71/document/orchestration/1485d8ae-277b-4fb5-8b42-9c002051d690","OutputS3BucketName":"","OutputS3KeyPrefix":"1485d8ae-277b-4fb5-8b42-9c002051d690/mi-0bda32df7d3617d71","CloudWatchConfig":{"LogGroupName":"/aws/ssm/AWS-RunShellScript","LogStreamPrefix":"1485d8ae-277b-4fb5-8b42-9c002051d690/mi-0bda32df7d3617d71","LogGroupEncryptionEnabled":false}}}
2020-06-23 14:34:41 INFO [MessageGatewayService] listening reply.
2020-06-23 14:34:41 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] Running plugin aws:runShellScript
2020-06-23 14:34:41 INFO [MessagingDeliveryService] [EngineProcessor] Processing in-progress document 1485d8ae-277b-4fb5-8b42-9c002051d690
2020-06-23 14:34:41 INFO [MessagingDeliveryService] Starting message polling
2020-06-23 14:34:41 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] [pluginName=aws:runShellScript] aws:runShellScript started with configuration {<nil> map[id:0.aws:runShellScript runCommand:[exit 194 echo "rebooted"] timeoutSeconds:3600 workingDirectory:] 1485d8ae-277b-4fb5-8b42-9c002051d690/mi-0bda32df7d3617d71/awsrunShellScript  false  false /var/lib/amazon/ssm/mi-0bda32df7d3617d71/document/orchestration/1485d8ae-277b-4fb5-8b42-9c002051d690/awsrunShellScript aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71 1485d8ae-277b-4fb5-8b42-9c002051d690 aws:runShellScript aws:runShellScript  map[] false []    false }
2020-06-23 14:34:41 INFO [MessagingDeliveryService] Starting send replies to MDS
2020-06-23 14:34:41 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] [pluginName=aws:runShellScript] The execution of command returned Exit Status: 194
2020-06-23 14:34:41 INFO [instanceID=mi-0bda32df7d3617d71] Starting association polling
2020-06-23 14:34:41 INFO [MessagingDeliveryService] [Association] [EngineProcessor] Starting
2020-06-23 14:34:41 INFO [MessagingDeliveryService] [Association] Launching response handler
2020-06-23 14:34:41 INFO [MessagingDeliveryService] [Association] [EngineProcessor] Initial processing
2020-06-23 14:34:41 INFO [OfflineService] Starting message polling
2020-06-23 14:34:41 INFO [OfflineService] Starting send replies to MDS
2020-06-23 14:34:41 INFO [MessagingDeliveryService] [Association] [EngineProcessor] Found in-progress document - 1485d8ae-277b-4fb5-8b42-9c002051d690
2020-06-23 14:34:41 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] channel: 1485d8ae-277b-4fb5-8b42-9c002051d690 not found, creating a new file channel...
2020-06-23 14:34:41 INFO [MessagingDeliveryService] [Association] Initializing association scheduling service
2020-06-23 14:34:41 INFO [MessagingDeliveryService] [Association] Association scheduling service initialized
2020-06-23 14:34:41 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] inter process communication started
2020-06-23 14:34:41 INFO [MessageGatewayService] Opening websocket connection to: wss://ssmmessages.eu-central-1.amazonaws.com/v1/control-channel/mi-0bda32df7d3617d71?role=subscribe&stream=input
2020-06-23 14:34:41 INFO [MessageGatewayService] Successfully opened websocket connection to: wss://ssmmessages.eu-central-1.amazonaws.com/v1/control-channel/mi-0bda32df7d3617d71?role=subscribe&stream=input
2020-06-23 14:34:41 INFO [MessageGatewayService] Starting receiving message from control channel
2020-06-23 14:34:41 INFO [MessageGatewayService] [EngineProcessor] Initial processing
2020-06-23 14:34:41 INFO [MessageGatewayService] [EngineProcessor] Found in-progress document - 1485d8ae-277b-4fb5-8b42-9c002051d690
2020-06-23 14:34:47 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] Sending plugin aws:runShellScript completion message
2020-06-23 14:34:47 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] requested terminate messaging worker, destroying the channel
2020-06-23 14:34:47 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] document execution complete
2020-06-23 14:34:47 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] sending document complete response...
2020-06-23 14:34:47 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] stopping ipc worker...
2020-06-23 14:34:47 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] requested shutdown, prepare to stop messaging
2020-06-23 14:34:47 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] channel /var/lib/amazon/ssm/mi-0bda32df7d3617d71/channels/1485d8ae-277b-4fb5-8b42-9c002051d690 requested close
2020-06-23 14:34:47 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] ipc channel closed, stop messaging worker
2020-06-23 14:34:47 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] channel /var/lib/amazon/ssm/mi-0bda32df7d3617d71/channels/1485d8ae-277b-4fb5-8b42-9c002051d690 closed
2020-06-23 14:34:47 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] document worker closed
2020-06-23 14:34:47 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] channel /var/lib/amazon/ssm/mi-0bda32df7d3617d71/channels/1485d8ae-277b-4fb5-8b42-9c002051d690 requested close
2020-06-23 14:34:47 INFO [MessagingDeliveryService] [EngineProcessor] sending reply for plugin update: aws:runShellScript
2020-06-23 14:34:47 INFO [MessagingDeliveryService] [EngineProcessor] sending document: 1485d8ae-277b-4fb5-8b42-9c002051d690 complete response
2020-06-23 14:34:47 INFO [MessagingDeliveryService] received plugin: aws:runShellScript result from Processor
2020-06-23 14:34:47 INFO [MessagingDeliveryService] Sending reply {
  "additionalInfo": {
    "agent": {
      "lang": "en-US",
      "name": "amazon-ssm-agent",
      "os": "",
      "osver": "1",
      "ver": ""
    },
    "dateTime": "2020-06-23T11:34:47.632Z",
    "runId": "",
    "runtimeStatusCounts": {
      "SuccessAndReboot": 1
    }
  },
  "documentStatus": "InProgress",
  "documentTraceOutput": "",
  "runtimeStatus": {
    "aws:runShellScript": {
      "status": "SuccessAndReboot",
      "code": 194,
      "name": "aws:runShellScript",
      "output": "",
      "startDateTime": "2020-06-23T11:30:42.012Z",
      "endDateTime": "2020-06-23T11:34:47.631Z",
      "outputS3BucketName": "",
      "outputS3KeyPrefix": "",
      "stepName": "",
      "standardOutput": "",
      "standardError": ""
    }
  }
}
2020-06-23 14:34:47 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] Executer closed
2020-06-23 14:34:47 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] channel /var/lib/amazon/ssm/mi-0bda32df7d3617d71/channels/1485d8ae-277b-4fb5-8b42-9c002051d690 closed
2020-06-23 14:34:47 INFO [MessagingDeliveryService] SendReply Response{
  Description: "Reply a0444fb3-1c00-4875-a86e-d60f15afe0b1 was successfully sent.",
  MessageId: "aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71",
  ReplyId: "a0444fb3-1c00-4875-a86e-d60f15afe0b1",
  ReplyStatus: "QUEUED"
}
2020-06-23 14:34:47 INFO [MessagingDeliveryService] command: aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71 complete
2020-06-23 14:34:47 INFO [MessagingDeliveryService] [EngineProcessor] document aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71 requested reboot, need to resume
2020-06-23 14:34:47 INFO [MessagingDeliveryService] [EngineProcessor] successfully requested a reboot
2020-06-23 14:34:47 INFO [instanceID=mi-0bda32df7d3617d71] A plugin has requested a reboot.
2020-06-23 14:34:47 INFO [instanceID=mi-0bda32df7d3617d71] Processing reboot request...
2020-06-23 14:34:47 INFO [instanceID=mi-0bda32df7d3617d71] core manager stop requested. Stop type: SoftStop
2020-06-23 14:34:47 INFO [instanceID=mi-0bda32df7d3617d71] Rebooting the machine in +1 Minutes..
2020-06-23 14:34:47 INFO [MessageGatewayService] Stopping MessageGatewayService.
2020-06-23 14:34:47 INFO [MessageGatewayService] Closing controlchannel with channel Id mi-0bda32df7d3617d71
2020-06-23 14:34:47 INFO [MessageGatewayService] Closing websocket channel connection to: wss://ssmmessages.eu-central-1.amazonaws.com/v1/control-channel/mi-0bda32df7d3617d71?role=subscribe&stream=input
2020-06-23 14:34:47 INFO [MessagingDeliveryService] Sending reply {
  "additionalInfo": {
    "agent": {
      "lang": "en-US",
      "name": "amazon-ssm-agent",
      "os": "",
      "osver": "1",
      "ver": ""
    },
    "dateTime": "2020-06-23T11:34:47.908Z",
    "runId": "",
    "runtimeStatusCounts": {
      "SuccessAndReboot": 1
    }
  },
  "documentStatus": "SuccessAndReboot",
  "documentTraceOutput": "",
  "runtimeStatus": {
    "aws:runShellScript": {
      "status": "SuccessAndReboot",
      "code": 194,
      "name": "aws:runShellScript",
      "output": "",
      "startDateTime": "2020-06-23T11:30:42.012Z",
      "endDateTime": "2020-06-23T11:34:47.631Z",
      "outputS3BucketName": "",
      "outputS3KeyPrefix": "",
      "stepName": "",
      "standardOutput": "",
      "standardError": ""
    }
  }
}
2020-06-23 14:34:47 INFO [MessageGatewayService] Ending the channel listening routine since the channel is closed
2020-06-23 14:34:47 INFO [MessageGatewayService] Successfully closed websocket connection to: 52.119.189.13:443
2020-06-23 14:34:47 INFO [instanceID=mi-0bda32df7d3617d71] shutdown output: 

2020-06-23 14:34:48 INFO [MessagingDeliveryService] SendReply Response{
  Description: "Reply 08c7b675-8b7a-44d0-ad6e-11b62ef6f3ba was successfully sent.",
  MessageId: "aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71",
  ReplyId: "08c7b675-8b7a-44d0-ad6e-11b62ef6f3ba",
  ReplyStatus: "QUEUED"
}
2020-06-23 14:34:53 INFO [MessagingDeliveryService] [Association] No associations on boot. Requerying for associations after 30 seconds.
2020-06-23 14:35:23 INFO [MessagingDeliveryService] [Association] Schedule manager refreshed with 0 associations, 0 new associations associated
2020-06-23 14:35:47 INFO Got signal:terminated value:0x563ff957d080
2020-06-23 14:35:47 INFO Stopping agent
2020-06-23 14:35:47 INFO [instanceID=mi-0bda32df7d3617d71] core manager stop requested. Stop type: HardStop
2020-06-23 14:35:47 INFO [MessageGatewayService] Stopping MessageGatewayService.
2020-06-23 14:35:47 INFO [MessageGatewayService] Closing controlchannel with channel Id mi-0bda32df7d3617d71
2020-06-23 14:35:47 INFO [MessageGatewayService] Closing websocket channel connection to: wss://ssmmessages.eu-central-1.amazonaws.com/v1/control-channel/mi-0bda32df7d3617d71?role=subscribe&stream=input
2020-06-23 14:35:52 INFO Bye.
2020-06-23 14:36:53 INFO Create new startup processor
2020-06-23 14:36:53 INFO [LongRunningPluginsManager] registered plugins: {}
2020-06-23 14:36:53 INFO Initializing bookkeeping folders
2020-06-23 14:36:53 INFO removing the completed state files
2020-06-23 14:36:53 INFO Initializing bookkeeping folders for long running plugins
2020-06-23 14:36:53 INFO Initializing replies folder for MDS reply requests that couldn't reach the service
2020-06-23 14:36:53 INFO Initializing healthcheck folders for long running plugins
2020-06-23 14:36:53 INFO Initializing locations for inventory plugin
2020-06-23 14:36:53 INFO Initializing default location for custom inventory
2020-06-23 14:36:53 INFO Initializing default location for file inventory
2020-06-23 14:36:53 INFO Initializing default location for role inventory
2020-06-23 14:36:53 INFO Init the cloudwatchlogs publisher
2020-06-23 14:36:53 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:downloadContent
2020-06-23 14:36:53 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:configureDocker
2020-06-23 14:36:53 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:runDockerAction
2020-06-23 14:36:53 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:refreshAssociation
2020-06-23 14:36:53 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:configurePackage
2020-06-23 14:36:53 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:runDocument
2020-06-23 14:36:53 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:softwareInventory
2020-06-23 14:36:53 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:runPowerShellScript
2020-06-23 14:36:53 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:updateSsmAgent
2020-06-23 14:36:53 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform dependent plugin aws:runShellScript
2020-06-23 14:36:53 INFO Starting Agent: amazon-ssm-agent - v2.3.722.0
2020-06-23 14:36:53 INFO OS: linux, Arch: amd64
2020-06-23 14:36:53 INFO parsing args: [/usr/bin/ssm-document-worker 1485d8ae-277b-4fb5-8b42-9c002051d690 mi-0bda32df7d3617d71]
2020-06-23 14:36:53 INFO [MessageGatewayService] Starting session document processing engine...
2020-06-23 14:36:53 INFO using channelName 1485d8ae-277b-4fb5-8b42-9c002051d690, instanceID: mi-0bda32df7d3617d71
2020-06-23 14:36:53 INFO [OfflineService] Starting document processing engine...
2020-06-23 14:36:53 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] document: 1485d8ae-277b-4fb5-8b42-9c002051d690 worker started
2020-06-23 14:36:53 INFO [OfflineService] [EngineProcessor] Starting
2020-06-23 14:36:53 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] channel: 1485d8ae-277b-4fb5-8b42-9c002051d690 found
2020-06-23 14:36:53 INFO [OfflineService] [EngineProcessor] Initial processing
2020-06-23 14:36:53 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:updateSsmAgent
2020-06-23 14:36:53 INFO [MessageGatewayService] [EngineProcessor] Starting
2020-06-23 14:36:53 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:configureDocker
2020-06-23 14:36:53 INFO [MessageGatewayService] SSM Agent is trying to setup control channel for Session Manager module.
2020-06-23 14:36:53 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:refreshAssociation
2020-06-23 14:36:53 INFO [HealthCheck] HealthCheck reporting agent health.
2020-06-23 14:36:53 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:configurePackage
2020-06-23 14:36:53 INFO [OfflineService] [EngineProcessor] Found in-progress document - 1485d8ae-277b-4fb5-8b42-9c002051d690
2020-06-23 14:36:53 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:downloadContent
2020-06-23 14:36:53 INFO [LongRunningPluginsManager] starting long running plugin manager
2020-06-23 14:36:53 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:softwareInventory
2020-06-23 14:36:53 INFO [MessageGatewayService] listening reply.
2020-06-23 14:36:53 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:runDockerAction
2020-06-23 14:36:53 INFO [MessagingDeliveryService] Starting document processing engine...
2020-06-23 14:36:53 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:runDocument
2020-06-23 14:36:53 INFO [MessagingDeliveryService] [EngineProcessor] Starting
2020-06-23 14:36:53 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:runPowerShellScript
2020-06-23 14:36:53 INFO [LongRunningPluginsManager] there aren't any long running plugin to execute
2020-06-23 14:36:53 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform dependent plugin aws:runShellScript
2020-06-23 14:36:53 INFO [MessageGatewayService] Setting up websocket for controlchannel for instance: mi-0bda32df7d3617d71, requestId: 97475f3c-f130-4d42-895e-f30ca2926773
2020-06-23 14:36:53 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] inter process communication started
2020-06-23 14:36:53 INFO [MessagingDeliveryService] [EngineProcessor] Initial processing
2020-06-23 14:36:53 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] received plugin config message
2020-06-23 14:36:53 INFO [MessagingDeliveryService] [EngineProcessor] Found in-progress document - 1485d8ae-277b-4fb5-8b42-9c002051d690
2020-06-23 14:36:53 INFO [LongRunningPluginsManager] There are no long running plugins currently getting executed - skipping their healthcheck
2020-06-23 14:36:53 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] {"DocumentInformation":{"DocumentID":"1485d8ae-277b-4fb5-8b42-9c002051d690","CommandID":"1485d8ae-277b-4fb5-8b42-9c002051d690","AssociationID":"","InstanceID":"mi-0bda32df7d3617d71","MessageID":"aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71","RunID":"2020-06-23T11-30-41.643Z","CreatedDate":"2020-06-23T11:30:41.574Z","DocumentName":"AWS-RunShellScript","DocumentVersion":"","DocumentStatus":"SuccessAndReboot","RunCount":3,"ProcInfo":{"Pid":1181,"StartTime":"2020-06-23T11:36:53.469081376Z"},"ClientId":"","RunAsUser":""},"DocumentType":"SendCommand","SchemaVersion":"1.2","InstancePluginsInformation":[{"Configuration":{"Settings":null,"Properties":[{"id":"0.aws:runShellScript","runCommand":["exit 194","echo \"rebooted\""],"timeoutSeconds":"3600","workingDirectory":""}],"OutputS3KeyPrefix":"1485d8ae-277b-4fb5-8b42-9c002051d690/mi-0bda32df7d3617d71/awsrunShellScript","OutputS3BucketName":"","S3EncryptionEnabled":false,"CloudWatchLogGroup":"","CloudWatchEncryptionEnabled":false,"OrchestrationDirectory":"/var/lib/amazon/ssm/mi-0bda32df7d3617d71/document/orchestration/1485d8ae-277b-4fb5-8b42-9c002051d690/awsrunShellScript","MessageId":"aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71","BookKeepingFileName":"1485d8ae-277b-4fb5-8b42-9c002051d690","PluginName":"aws:runShellScript","PluginID":"aws:runShellScript","DefaultWorkingDirectory":"","Preconditions":null,"IsPreconditionEnabled":false,"CurrentAssociations":null,"SessionId":"","ClientId":"","KmsKeyId":"","RunAsEnabled":false,"RunAsUser":""},"Name":"aws:runShellScript","Result":{"pluginID":"aws:runShellScript","pluginName":"aws:runShellScript","status":"SuccessAndReboot","code":194,"output":"","startDateTime":"2020-06-23T14:30:42.012557706+03:00","endDateTime":"2020-06-23T14:34:47.631299444+03:00","outputS3BucketName":"","outputS3KeyPrefix":"","stepName":"","error":"","standardOutput":"","standardError":""},"Id":"aws:runShellScript"}],"CancelInformation":{"CancelMessageID":"","CancelCommandID":"","Payload":"","DebugInfo":""},"IOConfig":{"OrchestrationDirectory":"/var/lib/amazon/ssm/mi-0bda32df7d3617d71/document/orchestration/1485d8ae-277b-4fb5-8b42-9c002051d690","OutputS3BucketName":"","OutputS3KeyPrefix":"1485d8ae-277b-4fb5-8b42-9c002051d690/mi-0bda32df7d3617d71","CloudWatchConfig":{"LogGroupName":"/aws/ssm/AWS-RunShellScript","LogStreamPrefix":"1485d8ae-277b-4fb5-8b42-9c002051d690/mi-0bda32df7d3617d71","LogGroupEncryptionEnabled":false}}}
2020-06-23 14:36:53 INFO [OfflineService] Starting message polling
2020-06-23 14:36:53 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] Running plugin aws:runShellScript
2020-06-23 14:36:53 INFO [OfflineService] Starting send replies to MDS
2020-06-23 14:36:54 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] [pluginName=aws:runShellScript] aws:runShellScript started with configuration {<nil> map[id:0.aws:runShellScript runCommand:[exit 194 echo "rebooted"] timeoutSeconds:3600 workingDirectory:] 1485d8ae-277b-4fb5-8b42-9c002051d690/mi-0bda32df7d3617d71/awsrunShellScript  false  false /var/lib/amazon/ssm/mi-0bda32df7d3617d71/document/orchestration/1485d8ae-277b-4fb5-8b42-9c002051d690/awsrunShellScript aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71 1485d8ae-277b-4fb5-8b42-9c002051d690 aws:runShellScript aws:runShellScript  map[] false []    false }
2020-06-23 14:36:53 INFO [MessagingDeliveryService] [EngineProcessor] Processing in-progress document 1485d8ae-277b-4fb5-8b42-9c002051d690
2020-06-23 14:36:54 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] [pluginName=aws:runShellScript] The execution of command returned Exit Status: 194
2020-06-23 14:36:53 INFO [MessagingDeliveryService] Starting message polling
2020-06-23 14:36:53 INFO [MessagingDeliveryService] Starting send replies to MDS
2020-06-23 14:36:53 INFO [instanceID=mi-0bda32df7d3617d71] Starting association polling
2020-06-23 14:36:53 INFO [MessagingDeliveryService] [Association] [EngineProcessor] Starting
2020-06-23 14:36:53 INFO [MessagingDeliveryService] [Association] Launching response handler
2020-06-23 14:36:53 INFO [MessagingDeliveryService] [Association] [EngineProcessor] Initial processing
2020-06-23 14:36:53 INFO [MessagingDeliveryService] [Association] [EngineProcessor] Found in-progress document - 1485d8ae-277b-4fb5-8b42-9c002051d690
2020-06-23 14:36:53 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] channel: 1485d8ae-277b-4fb5-8b42-9c002051d690 not found, creating a new file channel...
2020-06-23 14:36:53 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] inter process communication started
2020-06-23 14:36:53 INFO [MessagingDeliveryService] [Association] Initializing association scheduling service
2020-06-23 14:36:53 INFO [MessagingDeliveryService] [Association] Association scheduling service initialized
2020-06-23 14:36:53 INFO [MessageGatewayService] Opening websocket connection to: wss://ssmmessages.eu-central-1.amazonaws.com/v1/control-channel/mi-0bda32df7d3617d71?role=subscribe&stream=input
2020-06-23 14:36:53 INFO [MessageGatewayService] Successfully opened websocket connection to: wss://ssmmessages.eu-central-1.amazonaws.com/v1/control-channel/mi-0bda32df7d3617d71?role=subscribe&stream=input
2020-06-23 14:36:53 INFO [MessageGatewayService] Starting receiving message from control channel
2020-06-23 14:36:53 INFO [MessageGatewayService] [EngineProcessor] Initial processing
2020-06-23 14:36:53 INFO [MessageGatewayService] [EngineProcessor] Found in-progress document - 1485d8ae-277b-4fb5-8b42-9c002051d690

...

2020-06-23 15:01:16 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] Sending plugin aws:runShellScript completion message
2020-06-23 15:01:16 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] document execution complete
2020-06-23 15:01:16 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] sending document complete response...
2020-06-23 15:01:16 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] stopping ipc worker...
2020-06-23 15:01:16 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] requested shutdown, prepare to stop messaging
2020-06-23 15:01:16 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] channel /var/lib/amazon/ssm/mi-0bda32df7d3617d71/channels/1485d8ae-277b-4fb5-8b42-9c002051d690 requested close
2020-06-23 15:01:16 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] ipc channel closed, stop messaging worker
2020-06-23 15:01:16 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] document worker closed
2020-06-23 15:01:16 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] requested terminate messaging worker, destroying the channel
2020-06-23 15:01:16 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] channel /var/lib/amazon/ssm/mi-0bda32df7d3617d71/channels/1485d8ae-277b-4fb5-8b42-9c002051d690 requested close
2020-06-23 15:01:16 INFO [MessagingDeliveryService] [EngineProcessor] sending reply for plugin update: aws:runShellScript
2020-06-23 15:01:16 INFO [MessagingDeliveryService] [EngineProcessor] sending document: 1485d8ae-277b-4fb5-8b42-9c002051d690 complete response
2020-06-23 15:01:16 INFO [MessagingDeliveryService] received plugin: aws:runShellScript result from Processor
2020-06-23 15:01:16 INFO [MessagingDeliveryService] Sending reply {
  "additionalInfo": {
    "agent": {
      "lang": "en-US",
      "name": "amazon-ssm-agent",
      "os": "",
      "osver": "1",
      "ver": ""
    },
    "dateTime": "2020-06-23T12:01:16.783Z",
    "runId": "",
    "runtimeStatusCounts": {
      "SuccessAndReboot": 1
    }
  },
  "documentStatus": "InProgress",
  "documentTraceOutput": "",
  "runtimeStatus": {
    "aws:runShellScript": {
      "status": "SuccessAndReboot",
      "code": 194,
      "name": "aws:runShellScript",
      "output": "",
      "startDateTime": "2020-06-23T11:30:42.012Z",
      "endDateTime": "2020-06-23T12:01:16.780Z",
      "outputS3BucketName": "",
      "outputS3KeyPrefix": "",
      "stepName": "",
      "standardOutput": "",
      "standardError": ""
    }
  }
}
2020-06-23 15:01:16 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] channel /var/lib/amazon/ssm/mi-0bda32df7d3617d71/channels/1485d8ae-277b-4fb5-8b42-9c002051d690 closed
2020-06-23 15:01:16 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] Executer closed
2020-06-23 15:01:17 INFO [MessagingDeliveryService] SendReply Response{
  Description: "Reply a5ec14eb-4cd7-4a2d-9004-aeaae5109a6b was successfully sent.",
  MessageId: "aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71",
  ReplyId: "a5ec14eb-4cd7-4a2d-9004-aeaae5109a6b",
  ReplyStatus: "QUEUED"
}
2020-06-23 15:01:17 INFO [MessagingDeliveryService] command: aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71 complete
2020-06-23 15:01:17 INFO [MessagingDeliveryService] [EngineProcessor] document aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71 requested reboot, need to resume
2020-06-23 15:01:17 INFO [MessagingDeliveryService] [EngineProcessor] successfully requested a reboot
2020-06-23 15:01:17 INFO [instanceID=mi-0bda32df7d3617d71] A plugin has requested a reboot.
2020-06-23 15:01:17 INFO [instanceID=mi-0bda32df7d3617d71] Processing reboot request...
2020-06-23 15:01:17 INFO [instanceID=mi-0bda32df7d3617d71] core manager stop requested. Stop type: SoftStop
2020-06-23 15:01:17 INFO [MessageGatewayService] Stopping MessageGatewayService.
2020-06-23 15:01:17 INFO [MessageGatewayService] Closing controlchannel with channel Id mi-0bda32df7d3617d71
2020-06-23 15:01:17 INFO [MessageGatewayService] Closing websocket channel connection to: wss://ssmmessages.eu-central-1.amazonaws.com/v1/control-channel/mi-0bda32df7d3617d71?role=subscribe&stream=input
2020-06-23 15:01:17 INFO [MessagingDeliveryService] Sending reply {
  "additionalInfo": {
    "agent": {
      "lang": "en-US",
      "name": "amazon-ssm-agent",
      "os": "",
      "osver": "1",
      "ver": ""
    },
    "dateTime": "2020-06-23T12:01:17.136Z",
    "runId": "",
    "runtimeStatusCounts": {
      "SuccessAndReboot": 1
    }
  },
  "documentStatus": "SuccessAndReboot",
  "documentTraceOutput": "",
  "runtimeStatus": {
    "aws:runShellScript": {
      "status": "SuccessAndReboot",
      "code": 194,
      "name": "aws:runShellScript",
      "output": "",
      "startDateTime": "2020-06-23T11:30:42.012Z",
      "endDateTime": "2020-06-23T12:01:16.780Z",
      "outputS3BucketName": "",
      "outputS3KeyPrefix": "",
      "stepName": "",
      "standardOutput": "",
      "standardError": ""
    }
  }
}
2020-06-23 15:01:17 INFO [MessageGatewayService] Ending the channel listening routine since the channel is closed
2020-06-23 15:01:17 INFO [MessageGatewayService] Successfully closed websocket connection to: 52.94.205.120:443
2020-06-23 15:01:17 INFO [instanceID=mi-0bda32df7d3617d71] Rebooting the machine in +1 Minutes..
2020-06-23 15:01:17 INFO [instanceID=mi-0bda32df7d3617d71] shutdown output: 

2020-06-23 15:01:17 INFO [MessagingDeliveryService] SendReply Response{
  Description: "Reply 00430592-eb7b-418c-8501-fe62fe415e2f was successfully sent.",
  MessageId: "aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71",
  ReplyId: "00430592-eb7b-418c-8501-fe62fe415e2f",
  ReplyStatus: "QUEUED"
}
2020-06-23 15:01:29 INFO [MessagingDeliveryService] [Association] No associations on boot. Requerying for associations after 30 seconds.
2020-06-23 15:01:59 INFO [MessagingDeliveryService] [Association] Schedule manager refreshed with 0 associations, 0 new associations associated
2020-06-23 15:02:17 INFO Got signal:terminated value:0x563503be6080
2020-06-23 15:02:17 INFO Stopping agent
2020-06-23 15:02:17 INFO [instanceID=mi-0bda32df7d3617d71] core manager stop requested. Stop type: HardStop
2020-06-23 15:02:17 INFO [MessageGatewayService] Stopping MessageGatewayService.
2020-06-23 15:02:17 INFO [MessageGatewayService] Closing controlchannel with channel Id mi-0bda32df7d3617d71
2020-06-23 15:02:17 INFO [MessageGatewayService] Closing websocket channel connection to: wss://ssmmessages.eu-central-1.amazonaws.com/v1/control-channel/mi-0bda32df7d3617d71?role=subscribe&stream=input
2020-06-23 15:02:22 INFO Bye.
2020-06-23 15:03:23 INFO Create new startup processor
2020-06-23 15:03:23 INFO [LongRunningPluginsManager] registered plugins: {}
2020-06-23 15:03:23 INFO Initializing bookkeeping folders
2020-06-23 15:03:23 INFO removing the completed state files
2020-06-23 15:03:23 INFO Initializing bookkeeping folders for long running plugins
2020-06-23 15:03:23 INFO Initializing replies folder for MDS reply requests that couldn't reach the service
2020-06-23 15:03:23 INFO Initializing healthcheck folders for long running plugins
2020-06-23 15:03:23 INFO Initializing locations for inventory plugin
2020-06-23 15:03:23 INFO Initializing default location for custom inventory
2020-06-23 15:03:23 INFO Initializing default location for file inventory
2020-06-23 15:03:23 INFO Initializing default location for role inventory
2020-06-23 15:03:23 INFO Init the cloudwatchlogs publisher
2020-06-23 15:03:23 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:runPowerShellScript
2020-06-23 15:03:23 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:configureDocker
2020-06-23 15:03:23 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:runDockerAction
2020-06-23 15:03:23 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:runDocument
2020-06-23 15:03:23 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:softwareInventory
2020-06-23 15:03:23 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:updateSsmAgent
2020-06-23 15:03:23 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:refreshAssociation
2020-06-23 15:03:23 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:configurePackage
2020-06-23 15:03:23 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:downloadContent
2020-06-23 15:03:23 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform dependent plugin aws:runShellScript
2020-06-23 15:03:23 INFO Starting Agent: amazon-ssm-agent - v2.3.722.0
2020-06-23 15:03:23 INFO OS: linux, Arch: amd64
2020-06-23 15:03:23 INFO parsing args: [/usr/bin/ssm-document-worker 1485d8ae-277b-4fb5-8b42-9c002051d690 mi-0bda32df7d3617d71]
2020-06-23 15:03:23 INFO [MessagingDeliveryService] Starting document processing engine...
2020-06-23 15:03:23 INFO using channelName 1485d8ae-277b-4fb5-8b42-9c002051d690, instanceID: mi-0bda32df7d3617d71
2020-06-23 15:03:23 INFO [OfflineService] Starting document processing engine...
2020-06-23 15:03:23 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] document: 1485d8ae-277b-4fb5-8b42-9c002051d690 worker started
2020-06-23 15:03:23 INFO [OfflineService] [EngineProcessor] Starting
2020-06-23 15:03:23 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] channel: 1485d8ae-277b-4fb5-8b42-9c002051d690 found
2020-06-23 15:03:23 INFO [OfflineService] [EngineProcessor] Initial processing
2020-06-23 15:03:23 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:runDocument
2020-06-23 15:03:23 INFO [LongRunningPluginsManager] starting long running plugin manager
2020-06-23 15:03:23 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:softwareInventory
2020-06-23 15:03:23 INFO [MessagingDeliveryService] [EngineProcessor] Starting
2020-06-23 15:03:23 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:runPowerShellScript
2020-06-23 15:03:23 INFO [MessagingDeliveryService] [EngineProcessor] Initial processing
2020-06-23 15:03:23 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:updateSsmAgent
2020-06-23 15:03:23 INFO [HealthCheck] HealthCheck reporting agent health.
2020-06-23 15:03:23 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:configureDocker
2020-06-23 15:03:23 INFO [OfflineService] [EngineProcessor] Found in-progress document - 1485d8ae-277b-4fb5-8b42-9c002051d690
2020-06-23 15:03:23 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:refreshAssociation
2020-06-23 15:03:23 INFO [MessagingDeliveryService] [EngineProcessor] Found in-progress document - 1485d8ae-277b-4fb5-8b42-9c002051d690
2020-06-23 15:03:23 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:runDockerAction
2020-06-23 15:03:23 INFO [MessageGatewayService] Starting session document processing engine...
2020-06-23 15:03:23 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:configurePackage
2020-06-23 15:03:23 INFO [MessageGatewayService] [EngineProcessor] Starting
2020-06-23 15:03:23 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform independent plugin aws:downloadContent
2020-06-23 15:03:23 INFO [MessageGatewayService] SSM Agent is trying to setup control channel for Session Manager module.
2020-06-23 15:03:23 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] Successfully loaded platform dependent plugin aws:runShellScript
2020-06-23 15:03:23 INFO [MessageGatewayService] Setting up websocket for controlchannel for instance: mi-0bda32df7d3617d71, requestId: 6f75f2fa-cd5e-4ea0-871c-6a5587abdd53
2020-06-23 15:03:23 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] inter process communication started
2020-06-23 15:03:23 INFO [LongRunningPluginsManager] there aren't any long running plugin to execute
2020-06-23 15:03:23 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] received plugin config message
2020-06-23 15:03:23 INFO [MessageGatewayService] listening reply.
2020-06-23 15:03:23 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] {"DocumentInformation":{"DocumentID":"1485d8ae-277b-4fb5-8b42-9c002051d690","CommandID":"1485d8ae-277b-4fb5-8b42-9c002051d690","AssociationID":"","InstanceID":"mi-0bda32df7d3617d71","MessageID":"aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71","RunID":"2020-06-23T11-30-41.643Z","CreatedDate":"2020-06-23T11:30:41.574Z","DocumentName":"AWS-RunShellScript","DocumentVersion":"","DocumentStatus":"SuccessAndReboot","RunCount":15,"ProcInfo":{"Pid":1174,"StartTime":"2020-06-23T12:03:23.513648201Z"},"ClientId":"","RunAsUser":""},"DocumentType":"SendCommand","SchemaVersion":"1.2","InstancePluginsInformation":[{"Configuration":{"Settings":null,"Properties":[{"id":"0.aws:runShellScript","runCommand":["exit 194","echo \"rebooted\""],"timeoutSeconds":"3600","workingDirectory":""}],"OutputS3KeyPrefix":"1485d8ae-277b-4fb5-8b42-9c002051d690/mi-0bda32df7d3617d71/awsrunShellScript","OutputS3BucketName":"","S3EncryptionEnabled":false,"CloudWatchLogGroup":"","CloudWatchEncryptionEnabled":false,"OrchestrationDirectory":"/var/lib/amazon/ssm/mi-0bda32df7d3617d71/document/orchestration/1485d8ae-277b-4fb5-8b42-9c002051d690/awsrunShellScript","MessageId":"aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71","BookKeepingFileName":"1485d8ae-277b-4fb5-8b42-9c002051d690","PluginName":"aws:runShellScript","PluginID":"aws:runShellScript","DefaultWorkingDirectory":"","Preconditions":null,"IsPreconditionEnabled":false,"CurrentAssociations":null,"SessionId":"","ClientId":"","KmsKeyId":"","RunAsEnabled":false,"RunAsUser":""},"Name":"aws:runShellScript","Result":{"pluginID":"aws:runShellScript","pluginName":"aws:runShellScript","status":"SuccessAndReboot","code":194,"output":"","startDateTime":"2020-06-23T14:30:42.012557706+03:00","endDateTime":"2020-06-23T15:01:16.780104439+03:00","outputS3BucketName":"","outputS3KeyPrefix":"","stepName":"","error":"","standardOutput":"","standardError":""},"Id":"aws:runShellScript"}],"CancelInformation":{"CancelMessageID":"","CancelCommandID":"","Payload":"","DebugInfo":""},"IOConfig":{"OrchestrationDirectory":"/var/lib/amazon/ssm/mi-0bda32df7d3617d71/document/orchestration/1485d8ae-277b-4fb5-8b42-9c002051d690","OutputS3BucketName":"","OutputS3KeyPrefix":"1485d8ae-277b-4fb5-8b42-9c002051d690/mi-0bda32df7d3617d71","CloudWatchConfig":{"LogGroupName":"/aws/ssm/AWS-RunShellScript","LogStreamPrefix":"1485d8ae-277b-4fb5-8b42-9c002051d690/mi-0bda32df7d3617d71","LogGroupEncryptionEnabled":false}}}
2020-06-23 15:03:23 INFO [LongRunningPluginsManager] There are no long running plugins currently getting executed - skipping their healthcheck
2020-06-23 15:03:23 INFO [MessagingDeliveryService] [EngineProcessor] Processing in-progress document 1485d8ae-277b-4fb5-8b42-9c002051d690
2020-06-23 15:03:23 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] Running plugin aws:runShellScript
2020-06-23 15:03:23 INFO [OfflineService] Starting message polling
2020-06-23 15:03:24 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] [pluginName=aws:runShellScript] aws:runShellScript started with configuration {<nil> map[id:0.aws:runShellScript runCommand:[exit 194 echo "rebooted"] timeoutSeconds:3600 workingDirectory:] 1485d8ae-277b-4fb5-8b42-9c002051d690/mi-0bda32df7d3617d71/awsrunShellScript  false  false /var/lib/amazon/ssm/mi-0bda32df7d3617d71/document/orchestration/1485d8ae-277b-4fb5-8b42-9c002051d690/awsrunShellScript aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71 1485d8ae-277b-4fb5-8b42-9c002051d690 aws:runShellScript aws:runShellScript  map[] false []    false }
2020-06-23 15:03:23 INFO [MessagingDeliveryService] Starting message polling
2020-06-23 15:03:24 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] [pluginName=aws:runShellScript] The execution of command returned Exit Status: 194
2020-06-23 15:03:23 INFO [OfflineService] Starting send replies to MDS
2020-06-23 15:03:23 INFO [MessagingDeliveryService] Starting send replies to MDS
2020-06-23 15:03:23 INFO [instanceID=mi-0bda32df7d3617d71] Starting association polling
2020-06-23 15:03:23 INFO [MessagingDeliveryService] [Association] [EngineProcessor] Starting
2020-06-23 15:03:23 INFO [MessagingDeliveryService] [Association] Launching response handler
2020-06-23 15:03:23 INFO [MessagingDeliveryService] [Association] [EngineProcessor] Initial processing
2020-06-23 15:03:23 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] channel: 1485d8ae-277b-4fb5-8b42-9c002051d690 not found, creating a new file channel...
2020-06-23 15:03:23 INFO [MessagingDeliveryService] [Association] [EngineProcessor] Found in-progress document - 1485d8ae-277b-4fb5-8b42-9c002051d690
2020-06-23 15:03:23 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] inter process communication started
2020-06-23 15:03:23 INFO [MessagingDeliveryService] [Association] Initializing association scheduling service
2020-06-23 15:03:23 INFO [MessagingDeliveryService] [Association] Association scheduling service initialized
2020-06-23 15:03:23 INFO [MessageGatewayService] Opening websocket connection to: wss://ssmmessages.eu-central-1.amazonaws.com/v1/control-channel/mi-0bda32df7d3617d71?role=subscribe&stream=input
2020-06-23 15:03:23 INFO [MessageGatewayService] Successfully opened websocket connection to: wss://ssmmessages.eu-central-1.amazonaws.com/v1/control-channel/mi-0bda32df7d3617d71?role=subscribe&stream=input
2020-06-23 15:03:23 INFO [MessageGatewayService] Starting receiving message from control channel
2020-06-23 15:03:23 INFO [MessageGatewayService] [EngineProcessor] Initial processing
2020-06-23 15:03:30 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] Sending plugin aws:runShellScript completion message
2020-06-23 15:03:30 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] document execution complete
2020-06-23 15:03:30 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] sending document complete response...
2020-06-23 15:03:30 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] [DataBackend] stopping ipc worker...
2020-06-23 15:03:30 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] requested shutdown, prepare to stop messaging
2020-06-23 15:03:30 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] channel /var/lib/amazon/ssm/mi-0bda32df7d3617d71/channels/1485d8ae-277b-4fb5-8b42-9c002051d690 requested close
2020-06-23 15:03:30 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] channel /var/lib/amazon/ssm/mi-0bda32df7d3617d71/channels/1485d8ae-277b-4fb5-8b42-9c002051d690 closed
2020-06-23 15:03:30 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] ipc channel closed, stop messaging worker
2020-06-23 15:03:30 INFO [ssm-document-worker] [1485d8ae-277b-4fb5-8b42-9c002051d690] document worker closed
2020-06-23 15:03:30 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] requested terminate messaging worker, destroying the channel
2020-06-23 15:03:30 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] channel /var/lib/amazon/ssm/mi-0bda32df7d3617d71/channels/1485d8ae-277b-4fb5-8b42-9c002051d690 requested close
2020-06-23 15:03:30 INFO [MessagingDeliveryService] [EngineProcessor] sending reply for plugin update: aws:runShellScript
2020-06-23 15:03:30 INFO [MessagingDeliveryService] [EngineProcessor] sending document: 1485d8ae-277b-4fb5-8b42-9c002051d690 complete response
2020-06-23 15:03:30 INFO [MessagingDeliveryService] received plugin: aws:runShellScript result from Processor
2020-06-23 15:03:30 INFO [MessagingDeliveryService] Sending reply {
  "additionalInfo": {
    "agent": {
      "lang": "en-US",
      "name": "amazon-ssm-agent",
      "os": "",
      "osver": "1",
      "ver": ""
    },
    "dateTime": "2020-06-23T12:03:30.124Z",
    "runId": "",
    "runtimeStatusCounts": {
      "SuccessAndReboot": 1
    }
  },
  "documentStatus": "InProgress",
  "documentTraceOutput": "",
  "runtimeStatus": {
    "aws:runShellScript": {
      "status": "SuccessAndReboot",
      "code": 194,
      "name": "aws:runShellScript",
      "output": "",
      "startDateTime": "2020-06-23T11:30:42.012Z",
      "endDateTime": "2020-06-23T12:03:30.123Z",
      "outputS3BucketName": "",
      "outputS3KeyPrefix": "",
      "stepName": "",
      "standardOutput": "",
      "standardError": ""
    }
  }
}
2020-06-23 15:03:30 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] Executer closed
2020-06-23 15:03:30 INFO [MessagingDeliveryService] [EngineProcessor] [OutOfProcExecuter] [1485d8ae-277b-4fb5-8b42-9c002051d690] channel /var/lib/amazon/ssm/mi-0bda32df7d3617d71/channels/1485d8ae-277b-4fb5-8b42-9c002051d690 closed
2020-06-23 15:03:30 INFO [MessagingDeliveryService] SendReply Response{
  Description: "Reply f2c5590b-14a3-49e4-9021-54426d04ac34 was successfully sent.",
  MessageId: "aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71",
  ReplyId: "f2c5590b-14a3-49e4-9021-54426d04ac34",
  ReplyStatus: "QUEUED"
}
2020-06-23 15:03:30 INFO [MessagingDeliveryService] command: aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71 complete
2020-06-23 15:03:30 INFO [MessagingDeliveryService] [EngineProcessor] document aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71 requested reboot, need to resume
2020-06-23 15:03:30 INFO [MessagingDeliveryService] [EngineProcessor] successfully requested a reboot
2020-06-23 15:03:30 INFO [MessagingDeliveryService] Sending reply {
  "additionalInfo": {
    "agent": {
      "lang": "en-US",
      "name": "amazon-ssm-agent",
      "os": "",
      "osver": "1",
      "ver": ""
    },
    "dateTime": "2020-06-23T12:03:30.421Z",
    "runId": "",
    "runtimeStatusCounts": {
      "SuccessAndReboot": 1
    }
  },
  "documentStatus": "SuccessAndReboot",
  "documentTraceOutput": "",
  "runtimeStatus": {
    "aws:runShellScript": {
      "status": "SuccessAndReboot",
      "code": 194,
      "name": "aws:runShellScript",
      "output": "",
      "startDateTime": "2020-06-23T11:30:42.012Z",
      "endDateTime": "2020-06-23T12:03:30.123Z",
      "outputS3BucketName": "",
      "outputS3KeyPrefix": "",
      "stepName": "",
      "standardOutput": "",
      "standardError": ""
    }
  }
}
2020-06-23 15:03:30 INFO [instanceID=mi-0bda32df7d3617d71] A plugin has requested a reboot.
2020-06-23 15:03:30 INFO [instanceID=mi-0bda32df7d3617d71] Processing reboot request...
2020-06-23 15:03:30 INFO [instanceID=mi-0bda32df7d3617d71] core manager stop requested. Stop type: SoftStop
2020-06-23 15:03:30 INFO [instanceID=mi-0bda32df7d3617d71] Rebooting the machine in +1 Minutes..
2020-06-23 15:03:30 INFO [MessageGatewayService] Stopping MessageGatewayService.
2020-06-23 15:03:30 INFO [MessageGatewayService] Closing controlchannel with channel Id mi-0bda32df7d3617d71
2020-06-23 15:03:30 INFO [MessageGatewayService] Closing websocket channel connection to: wss://ssmmessages.eu-central-1.amazonaws.com/v1/control-channel/mi-0bda32df7d3617d71?role=subscribe&stream=input
2020-06-23 15:03:30 INFO [MessageGatewayService] Ending the channel listening routine since the channel is closed
2020-06-23 15:03:30 INFO [MessageGatewayService] Successfully closed websocket connection to: 52.119.188.27:443
2020-06-23 15:03:30 INFO [instanceID=mi-0bda32df7d3617d71] shutdown output: 

2020-06-23 15:03:30 INFO [MessagingDeliveryService] SendReply Response{
  Description: "Reply a70841e1-3d90-46fb-bb11-0332c8feaa42 was successfully sent.",
  MessageId: "aws.ssm.1485d8ae-277b-4fb5-8b42-9c002051d690.mi-0bda32df7d3617d71",
  ReplyId: "a70841e1-3d90-46fb-bb11-0332c8feaa42",
  ReplyStatus: "QUEUED"
}
2020-06-23 15:03:50 INFO [MessagingDeliveryService] [Association] No associations on boot. Requerying for associations after 30 seconds.
2020-06-23 15:04:20 INFO [MessagingDeliveryService] [Association] Schedule manager refreshed with 0 associations, 0 new associations associated
2020-06-23 15:04:30 INFO Got signal:terminated value:0x557340bb5080
2020-06-23 15:04:30 INFO Stopping agent
2020-06-23 15:04:30 INFO [instanceID=mi-0bda32df7d3617d71] core manager stop requested. Stop type: HardStop
2020-06-23 15:04:30 INFO [MessageGatewayService] Stopping MessageGatewayService.
2020-06-23 15:04:30 INFO [MessageGatewayService] Closing controlchannel with channel Id mi-0bda32df7d3617d71
2020-06-23 15:04:30 INFO [MessageGatewayService] Closing websocket channel connection to: wss://ssmmessages.eu-central-1.amazonaws.com/v1/control-channel/mi-0bda32df7d3617d71?role=subscribe&stream=input
2020-06-23 15:04:35 INFO Bye.
2020-06-23 15:05:35 INFO Create new startup processor
2020-06-23 15:05:35 INFO [LongRunningPluginsManager] registered plugins: {}
2020-06-23 15:05:35 INFO Initializing bookkeeping folders
2020-06-23 15:05:35 INFO removing the completed state files
2020-06-23 15:05:35 INFO Initializing bookkeeping folders for long running plugins
2020-06-23 15:05:35 INFO Initializing replies folder for MDS reply requests that couldn't reach the service
2020-06-23 15:05:35 INFO Initializing healthcheck folders for long running plugins
2020-06-23 15:05:35 INFO Initializing locations for inventory plugin
2020-06-23 15:05:35 INFO Initializing default location for custom inventory
2020-06-23 15:05:35 INFO Initializing default location for file inventory
2020-06-23 15:05:35 INFO Initializing default location for role inventory
2020-06-23 15:05:35 INFO Init the cloudwatchlogs publisher
2020-06-23 15:05:35 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:softwareInventory
2020-06-23 15:05:35 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:runPowerShellScript
2020-06-23 15:05:35 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:runDockerAction
2020-06-23 15:05:35 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:configurePackage
2020-06-23 15:05:35 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:downloadContent
2020-06-23 15:05:35 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:runDocument
2020-06-23 15:05:35 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:updateSsmAgent
2020-06-23 15:05:35 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:configureDocker
2020-06-23 15:05:35 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform independent plugin aws:refreshAssociation
2020-06-23 15:05:35 INFO [instanceID=mi-0bda32df7d3617d71] Successfully loaded platform dependent plugin aws:runShellScript
2020-06-23 15:05:35 INFO Starting Agent: amazon-ssm-agent - v2.3.722.0
2020-06-23 15:05:35 INFO OS: linux, Arch: amd64
2020-06-23 15:05:35 INFO [MessagingDeliveryService] Starting document processing engine...
2020-06-23 15:05:35 INFO [MessagingDeliveryService] [EngineProcessor] Starting
2020-06-23 15:05:35 INFO [MessagingDeliveryService] [EngineProcessor] Initial processing
2020-06-23 15:05:35 INFO [OfflineService] Starting document processing engine...
2020-06-23 15:05:35 INFO [OfflineService] [EngineProcessor] Starting
2020-06-23 15:05:35 INFO [OfflineService] [EngineProcessor] Initial processing
2020-06-23 15:05:35 INFO [MessagingDeliveryService] [EngineProcessor] Found in-progress document - 1485d8ae-277b-4fb5-8b42-9c002051d690
2020-06-23 15:05:35 INFO [LongRunningPluginsManager] starting long running plugin manager
2020-06-23 15:05:35 INFO [OfflineService] [EngineProcessor] Found in-progress document - 1485d8ae-277b-4fb5-8b42-9c002051d690
2020-06-23 15:05:35 INFO [MessageGatewayService] Starting session document processing engine...
2020-06-23 15:05:35 INFO [MessageGatewayService] [EngineProcessor] Starting
2020-06-23 15:05:35 INFO [MessageGatewayService] SSM Agent is trying to setup control channel for Session Manager module.
2020-06-23 15:05:35 INFO [MessageGatewayService] Setting up websocket for controlchannel for instance: mi-0bda32df7d3617d71, requestId: 0c2d579a-c0f2-42af-810b-40aa58cc5d7c
2020-06-23 15:05:35 INFO [HealthCheck] HealthCheck reporting agent health.
2020-06-23 15:05:35 INFO [MessageGatewayService] listening reply.
2020-06-23 15:05:35 INFO [LongRunningPluginsManager] there aren't any long running plugin to execute
2020-06-23 15:05:35 INFO [LongRunningPluginsManager] There are no long running plugins currently getting executed - skipping their healthcheck
2020-06-23 15:05:35 INFO [OfflineService] Starting message polling
2020-06-23 15:05:35 INFO [OfflineService] Starting send replies to MDS
2020-06-23 15:05:35 INFO [MessagingDeliveryService] Starting message polling
2020-06-23 15:05:35 INFO [MessagingDeliveryService] Starting send replies to MDS
2020-06-23 15:05:35 INFO [instanceID=mi-0bda32df7d3617d71] Starting association polling
2020-06-23 15:05:35 INFO [MessagingDeliveryService] [Association] [EngineProcessor] Starting
2020-06-23 15:05:35 INFO [MessagingDeliveryService] [Association] Launching response handler
2020-06-23 15:05:35 INFO [MessagingDeliveryService] [Association] [EngineProcessor] Initial processing
2020-06-23 15:05:35 INFO [MessagingDeliveryService] [Association] Initializing association scheduling service
2020-06-23 15:05:35 INFO [MessagingDeliveryService] [Association] Association scheduling service initialized
2020-06-23 15:05:36 INFO [MessageGatewayService] Opening websocket connection to: wss://ssmmessages.eu-central-1.amazonaws.com/v1/control-channel/mi-0bda32df7d3617d71?role=subscribe&stream=input
2020-06-23 15:05:36 INFO [MessageGatewayService] Successfully opened websocket connection to: wss://ssmmessages.eu-central-1.amazonaws.com/v1/control-channel/mi-0bda32df7d3617d71?role=subscribe&stream=input
2020-06-23 15:05:36 INFO [MessageGatewayService] Starting receiving message from control channel
2020-06-23 15:05:36 INFO [MessageGatewayService] [EngineProcessor] Initial processing
2020-06-23 15:05:54 INFO [MessagingDeliveryService] [Association] No associations on boot. Requerying for associations after 30 seconds.
2020-06-23 15:05:55 INFO [HealthCheck] HealthCheck reporting agent health.
2020-06-23 15:06:24 INFO [MessagingDeliveryService] [Association] Schedule manager refreshed with 0 associations, 0 new associations associated
2020-06-23 15:10:55 INFO [HealthCheck] HealthCheck reporting agent health.
2020-06-23 15:15:44 INFO [MessagingDeliveryService] [Association] Schedule manager refreshed with 0 associations, 0 new associations associated
2020-06-23 15:15:55 INFO [HealthCheck] HealthCheck reporting agent health.
2020-06-23 15:20:35 INFO [LongRunningPluginsManager] There are no long running plugins currently getting executed - skipping their healthcheck
2020-06-23 15:20:55 INFO [HealthCheck] HealthCheck reporting agent health.
2020-06-23 15:25:55 INFO [HealthCheck] HealthCheck reporting agent health.
2020-06-23 15:26:00 INFO [MessagingDeliveryService] [Association] Schedule manager refreshed with 0 associations, 0 new associations associated
2020-06-23 15:30:55 INFO [HealthCheck] HealthCheck reporting agent health.
2020-06-23 15:35:35 INFO [LongRunningPluginsManager] There are no long running plugins currently getting executed - skipping their healthcheck
2020-06-23 15:35:49 INFO [MessagingDeliveryService] [Association] Schedule manager refreshed with 0 associations, 0 new associations associated
2020-06-23 15:35:55 INFO [HealthCheck] HealthCheck reporting agent health.
2020-06-23 15:40:55 INFO [HealthCheck] HealthCheck reporting agent health.
2020-06-23 15:45:36 INFO [MessagingDeliveryService] [Association] Schedule manager refreshed with 0 associations, 0 new associations associated
2020-06-23 15:45:55 INFO [HealthCheck] HealthCheck reporting agent health.
2020-06-23 15:50:10 INFO [MessageGatewayService] Validating SSM parameters
2020-06-23 15:50:10 INFO [ssm-session-worker] [root-06c5d17a3df79f6fa] document: root-06c5d17a3df79f6fa worker started
2020-06-23 15:50:10 INFO [MessageGatewayService] [EngineProcessor] [OutOfProcExecuter] [root-06c5d17a3df79f6fa] channel: root-06c5d17a3df79f6fa not found, creating a new file channel...
2020-06-23 15:50:10 INFO [ssm-session-worker] [root-06c5d17a3df79f6fa] channel: root-06c5d17a3df79f6fa found
2020-06-23 15:50:10 INFO [MessageGatewayService] [EngineProcessor] [OutOfProcExecuter] [root-06c5d17a3df79f6fa] inter process communication started
2020-06-23 15:50:10 INFO [ssm-session-worker] [root-06c5d17a3df79f6fa] inter process communication started
2020-06-23 15:50:10 INFO [ssm-session-worker] [root-06c5d17a3df79f6fa] [DataBackend] received plugin config message
2020-06-23 15:50:10 INFO [ssm-session-worker] [root-06c5d17a3df79f6fa] [DataBackend] {"DocumentInformation":{"DocumentID":"root-06c5d17a3df79f6fa","CommandID":"root-06c5d17a3df79f6fa","AssociationID":"","InstanceID":"mi-0bda32df7d3617d71","MessageID":"root-06c5d17a3df79f6fa","RunID":"2020-06-23T12-50-10.310Z","CreatedDate":"52447-07-08 22:11:47 +0200 EET","DocumentName":"","DocumentVersion":"","DocumentStatus":"InProgress","RunCount":0,"ProcInfo":{"Pid":5522,"StartTime":"2020-06-23T12:50:10.313273847Z"},"ClientId":"","RunAsUser":""},"DocumentType":"StartSession","SchemaVersion":"1.0","InstancePluginsInformation":[{"Configuration":{"Settings":null,"Properties":null,"OutputS3KeyPrefix":"","OutputS3BucketName":"","S3EncryptionEnabled":true,"CloudWatchLogGroup":"","CloudWatchEncryptionEnabled":true,"OrchestrationDirectory":"/var/lib/amazon/ssm/mi-0bda32df7d3617d71/session/orchestration/root-06c5d17a3df79f6fa/Standard_Stream","MessageId":"root-06c5d17a3df79f6fa","BookKeepingFileName":"root-06c5d17a3df79f6fa","PluginName":"Standard_Stream","PluginID":"Standard_Stream","DefaultWorkingDirectory":"","Preconditions":null,"IsPreconditionEnabled":false,"CurrentAssociations":null,"SessionId":"root-06c5d17a3df79f6fa","ClientId":"","KmsKeyId":"","RunAsEnabled":false,"RunAsUser":""},"Name":"Standard_Stream","Result":{"pluginID":"","pluginName":"","status":"","code":0,"output":null,"startDateTime":"0001-01-01T00:00:00Z","endDateTime":"0001-01-01T00:00:00Z","outputS3BucketName":"","outputS3KeyPrefix":"","stepName":"","error":"","standardOutput":"","standardError":""},"Id":"Standard_Stream"}],"CancelInformation":{"CancelMessageID":"","CancelCommandID":"","Payload":"","DebugInfo":""},"IOConfig":{"OrchestrationDirectory":"/var/lib/amazon/ssm/mi-0bda32df7d3617d71/session/orchestration/root-06c5d17a3df79f6fa","OutputS3BucketName":"","OutputS3KeyPrefix":"","CloudWatchConfig":{"LogGroupName":"","LogStreamPrefix":"","LogGroupEncryptionEnabled":false}}}
2020-06-23 15:50:10 INFO [ssm-session-worker] [root-06c5d17a3df79f6fa] [DataBackend] Running plugin Standard_Stream
2020-06-23 15:50:10 INFO [ssm-session-worker] [root-06c5d17a3df79f6fa] [DataBackend] [pluginName=Standard_Stream] Setting up datachannel for session: root-06c5d17a3df79f6fa, requestId: e4dab44a-ee46-4002-b9fe-0f67271fabbd, clientId: 
2020-06-23 15:50:11 INFO [ssm-session-worker] [root-06c5d17a3df79f6fa] [DataBackend] [pluginName=Standard_Stream] Opening websocket connection to: wss://ssmmessages.eu-central-1.amazonaws.com/v1/data-channel/root-06c5d17a3df79f6fa?role=publish_subscribe
2020-06-23 15:50:11 INFO [ssm-session-worker] [root-06c5d17a3df79f6fa] [DataBackend] [pluginName=Standard_Stream] Successfully opened websocket connection to: wss://ssmmessages.eu-central-1.amazonaws.com/v1/data-channel/root-06c5d17a3df79f6fa?role=publish_subscribe
2020-06-23 15:50:11 INFO [ssm-session-worker] [root-06c5d17a3df79f6fa] [DataBackend] [pluginName=Standard_Stream] Skipping handshake.
2020-06-23 15:50:11 INFO [ssm-session-worker] [root-06c5d17a3df79f6fa] [DataBackend] [pluginName=Standard_Stream] Starting pty
2020-06-23 15:50:11 INFO [ssm-session-worker] [root-06c5d17a3df79f6fa] [DataBackend] [pluginName=Standard_Stream] ssm-user already exists.
2020-06-23 15:50:11 INFO [ssm-session-worker] [root-06c5d17a3df79f6fa] [DataBackend] [pluginName=Standard_Stream] Plugin Standard_Stream started
2020-06-23 15:50:35 INFO [LongRunningPluginsManager] There are no long running plugins currently getting executed - skipping their healthcheck
2020-06-23 15:50:55 INFO [HealthCheck] HealthCheck reporting agent health.
2020-06-23 15:55:55 INFO [HealthCheck] HealthCheck reporting agent health.
2020-06-23 15:56:00 INFO [MessagingDeliveryService] [Association] Schedule manager refreshed with 0 associations, 0 new associations associated
2020-06-23 16:00:55 INFO [HealthCheck] HealthCheck reporting agent health.
2020-06-23 16:05:35 INFO [LongRunningPluginsManager] There are no long running plugins currently getting executed - skipping their healthcheck
2020-06-23 16:05:50 INFO [MessagingDeliveryService] [Association] Schedule manager refreshed with 0 associations, 0 new associations associated
2020-06-23 16:05:55 INFO [HealthCheck] HealthCheck reporting agent health.
2020-06-23 16:10:55 INFO [HealthCheck] HealthCheck reporting agent health.
2020-06-23 16:15:36 INFO [MessagingDeliveryService] [Association] Schedule manager refreshed with 0 associations, 0 new associations associated
2020-06-23 16:15:55 INFO [HealthCheck] HealthCheck reporting agent health.
Thor-Bjorgvinsson commented 4 years ago

Could you try updating your agent to latest and see if you still encounter this issue?

dorintt commented 4 years ago

I updated to 2.3.1319.0 and no difference.

But I think I found a workaround, using this script instead of simply calling exit 194:

#!/bin/sh

UPTIME_SECONDS=`awk '{print $1}' /proc/uptime`
UPTIME_SECONDS=${UPTIME_SECONDS%.*}

echo "uptime: $UPTIME_SECONDS seconds"

if [ $UPTIME_SECONDS -gt 60 ]; then
        echo "rebooting ..."
        exit 194
fi

echo "done"

This reboots only once, completes successfully and the logs appear on CloudWatch:

- 2020-06-26T12:46:11.828+03:00 | uptime: 2453 seconds rebooting ...
- 2020-06-26T12:48:10.795+03:00 | uptime: 2453 seconds rebooting ... uptime: 10 seconds done
dorintt commented 4 years ago

There is probably no issue with amazon-ssm-agent, just that the console documentation was not clear enough. After reboot, the shell script is actually re-run from the beginning (not from where it exited with 194) and it is user's responsibility designing the script to detect if the reboot is required again or not.

My detection method above is checking if the system was rebooted within the last 60 seconds, and skip the reboot if so.