search

aws / amazon-ssm-agent

An agent to enable remote management of your EC2 instances, on-premises servers, or virtual machines (VMs).
https://aws.amazon.com/systems-manager/
Apache License 2.0
1.03k stars 323 forks source link

aws ssm start-session fails with TargetNotConnected on ECS on Fargate #540

Open ganeshgk opened 8 months ago

ganeshgk commented 8 months ago

I have ECS running with Fargate launchtype: the containers are running successfully, however to troubleshoot/debug an issue I tried the AWS ECS Exec, however while trying with the https://github.com/aws-containers/amazon-ecs-exec-checker all checks are passed, it only shows in red 

 Managed Agent Status
    ----------
         1. STOPPED (Reason: null) for "containernameredacted" - LastStartedAt: null

Since there are many reports of ECS Exec not working & based on this comment https://github.com/aws/amazon-ssm-agent/issues/361#issuecomment-1007335151 I tried using the

aws ssm start-session --target ecs:.... but it throws TargetNotConnected error, 


aws ssm start-session --target ecs:clusternameredacted_e7e36d0d4e744104bd4d225c44daddfe_e7e36d0d4e744104bd4d225c44daddfe-3839356491

An error occurred (TargetNotConnected) when calling the StartSession operation: ecs:clusternameredacted_e7e36d0d4e744104bd4d225c44daddfe_e7e36d0d4e744104bd4d225c44daddfe-3839356491 is not connected

so my questions is

bcmedeiros commented 8 months ago

I struggled with a similar problem for hours, and running the ECS task with --enable-execute-command fixed the problem for me. ECS services can also be configured to spawn their tasks with that flag on.

This is really weird since nothing on https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-troubleshooting.html mentioned this flag, and SSM and ECS Exec not exactly the same thing AFAICS.

Anyway, I hope this helps.

ziwangj commented 8 months ago

Thanks for reaching out. Did above answer solve the issue you had? If not, can you please provide the below information for further investigation?

  1. The agent and SSMCLI version: https://docs.aws.amazon.com/systems-manager/latest/userguide/plugin-version-history.html https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-agent-get-version.html
  2. The agent log: https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-agent-logs.html