Support Credentials from IAM Roles Anywhere

[garciap2g]

Is your feature request related to a problem? Please describe. When I using credentials from IAM Roles Anywhere, I'm unable to use Cloudwatch Agent, since in OnPrem mode it looks for hard-coded Access Keys in the .aws config files

Describe the solution you'd like I'd like to use the standard IAM Roles Anywhere service, or the credential_process.

Additional context I've persued the "RUN_WITH_IRSA" config mode, but this is very poorly documented and also doesn't seem to solve the issue.

[chadpatel]

Thanks, created a ticket on our internal ticket tracker.

Can you share logs or any artifacts showing it not working as you expect so we can reproduce and triage

[nir-zilberman]

A similar request was also mentioned here: https://github.com/aws/amazon-cloudwatch-agent/issues/140#issuecomment-2140446258

[kensantoso]

Here are the logs if I set credential_process for the[AmazonCloudWatchAgent] profile in config(using iam roles anywhere creds helper)

$ cat /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log

2024/09/05 20:27:55 I! Changing ownership of [/opt/aws/amazon-cloudwatch-agent/logs /opt/aws/amazon-cloudwatch-agent/etc /opt/aws/amazon-cloudwatch-agent/var] to 0:0
2024-09-05T10:27:55Z I! Starting AmazonCloudWatchAgent CWAgent/1.300044.0b793 (go1.22.6; linux; amd64) with log file /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log with log target lumberjack
2024-09-05T10:27:55Z I! AWS SDK log level not set
2024-09-05T10:27:55Z I! creating new logs agent
2024-09-05T10:27:55Z I! [logagent] starting
2024-09-05T10:27:55Z I! {"caller":"service@v0.103.0/service.go:115","msg":"Setting up own telemetry..."}
2024-09-05T10:27:55Z I! {"caller":"service@v0.103.0/service.go:156","msg":"Skipped telemetry setup.","address":"","level":"None"}
2024-09-05T10:27:55Z I! {"caller":"service@v0.103.0/service.go:182","msg":"Starting CWAgent...","Version":"1.300044.0b793","NumCPU":6}
2024-09-05T10:27:55Z I! {"caller":"extensions/extensions.go:34","msg":"Starting extensions..."}
2024-09-05T10:27:55Z I! {"caller":"extensions/extensions.go:37","msg":"Extension is starting...","kind":"extension","name":"agenthealth/metrics"}
2024-09-05T10:27:55Z I! {"caller":"extensions/extensions.go:52","msg":"Extension started.","kind":"extension","name":"agenthealth/metrics"}
2024-09-05T10:27:55Z I! will use file based credentials provider
2024-09-05T10:27:55Z E! Failed to get credential from session: SharedCredsLoad: failed to get profile
2024-09-05T10:27:55Z I! cloudwatch: get unique roll up list []
2024-09-05T10:27:55Z I! cloudwatch: publish with ForceFlushInterval: 1m0s, Publish Jitter: 21.429439648s
2024-09-05T10:27:55Z I! Started the statsd service on :8125
2024-09-05T10:27:55Z I! [inputs.socket_listener] Listening on udp://127.0.0.1:25826
2024-09-05T10:27:55Z I! Statsd listener listening on:  [::]:8125
2024-09-05T10:27:55Z I! {"caller":"service@v0.103.0/service.go:208","msg":"Everything is ready. Begin running and processing data."}
2024-09-05T10:27:55Z W! {"caller":"localhostgate/featuregate.go:63","msg":"The default endpoints for all servers in components will change to use localhost instead of 0.0.0.0 in a future version. Use the feature gate to preview the new defa>
2024-09-05T10:29:21Z E! cloudwatch: code: SharedCredsLoad, message: failed to get profile, original error: <nil>
2024-09-05T10:29:21Z W! cloudwatch: 0 retries, going to sleep 191 ms before retrying.
2024-09-05T10:29:21Z E! cloudwatch: WriteToCloudWatch failure, err:  SharedCredsLoad: failed to get profile
2024-09-05T10:30:21Z E! cloudwatch: code: SharedCredsLoad, message: failed to get profile, original error: <nil>

The same error just repeats.