search

aws / amazon-ssm-agent

An agent to enable remote management of your EC2 instances, on-premises servers, or virtual machines (VMs).
https://aws.amazon.com/systems-manager/
Apache License 2.0
1.03k stars 323 forks source link

SSM Node not visible for hours after internet connectivity restored #479

Closed tdekoning93 closed 10 months ago

tdekoning93 commented 1 year ago

Hi,

This is mainly a question which I couldn't figure out from the documentation. Whenever we put a new node on a site, it will usually spin up without internet connectivity. This means Amazon SSM agent on Windows and Linux is already running without internet connectivity and therefore also shows up offline in the AWS Console. That makes sense.

When we however restore internet connectivity, it takes a very long time for the node to show up in the console as online. Usually multiple hours even. Restarting the service fixes this immediately. I guess the AWS SSM agent only polls AWS once every X amount of time, but I could only find the HealthFrequencyMinutes that looks similar to that, which is set to 5 minutes by default already.

Is there anything I'm missing that can be configured for the SSM agent to poll more often so we don't have to wait for hours, but instead when connectivity is restored we see it back online in minutes?

Hope this was clear. Thanks in advance!

roman-frei-89 commented 1 year ago

@tdekoning93 did you find a solution for the problem? I'm currently struggling with similar issues.

anujkumar-df commented 1 year ago

I think it uses an exponential backoff policy.

sluggard76 commented 10 months ago

tdekoning93,

It is mainly due to the exponential backoff policy. On the other hand, we have a feature request to make such policy configurable including an upper limits. We will be working on the request according to our plan. Will provide an ECD as soon as it is available.

doubleopinter commented 10 months ago

I would like to +1 this somehow, we run into this issue all the time with hybrid instances and it's really annoying. I just want to be able to set some upper limit like 1hr and be done with it.