search

aws / amazon-cloudwatch-agent

CloudWatch Agent enables you to collect and export host-level metrics and logs on instances running Linux or Windows server.
MIT License
448 stars 205 forks source link

Exit code 2 when Cloudwatch agent is almost immediately stopped #770

Open SamStephens opened 1 year ago

SamStephens commented 1 year ago

Describe the bug

I'm using Cloudwatch Agent as an ECS sidecar. I have a scenario where I spin up an ECS instance, and it almost immediately spins down doing no work and with no metrics emitted to the Cloudwatch Agent.

What I'm seeing is that when this happens, the Cloudwatch Agent Logstream is empty, and the Cloudwatch Agent container exits with Exit Code 2, causing my ECS task to be considered to have failed.

Steps to reproduce

Spin up an ECS instance with the Cloudwatch Agent as an ECS sidecar, and code that exits immediately with no metrics emitted to the Cloudwatch Agent.

What did you expect to see?

Cloudwatch Agent exit with exit code 0. Some logs indicating what the Agent has done.

What did you see instead?

Cloudwatch Agent exit with exit code 2. No logs indicating what the Agent has done.

What version did you use?

The current version from https://hub.docker.com/r/amazon/cloudwatch-agent, which appears to be 1.247359.1b252618.

What config did you use?

Config: { "logs": { "metrics_collected": { "emf": {}, } } }

Environment

ECS sidecar

Additional context

NA

github-actions[bot] commented 1 year ago

This issue was marked stale due to lack of activity.

SamStephens commented 1 year ago

Still relevant.

sethAmazon commented 1 year ago

Thank you for reporting the issue

Are you saying you have a container (your app) and a side car (cloudwatch agent)? Then, when you start the task, the app shuts down immediately, causing the cwa to exit before sending metrics to cw?

SamStephens commented 1 year ago

Are you saying you have a container (your app) and a side car (cloudwatch agent)? Then, when you start the task, the app shuts down immediately, causing the cwa to exit before sending metrics to cw?

@sethAmazon Correct, except that no metrics are reported to CWA, so the CWA does not exit before sending the metrics.

The problem isn't the lack of metrics emission - there's none to emit. The problem is that the sidecar exits with exit code 2 and causes my ECS task to be considered to be failed (this is running as part of a step function).

github-actions[bot] commented 3 months ago

This issue was marked stale due to lack of activity.

SamStephens commented 3 months ago

@sethAmazon does anyone still work at AWS?

github-actions[bot] commented 22 hours ago

This issue was marked stale due to lack of activity.

SamStephens commented 22 hours ago

Looks like there's still no one left working at AWS to pay attention to this issue 😑