search

grafana / cloudcost-exporter

Prometheus Exporter for Cloud Provider agnostic cost metrics
Apache License 2.0
30 stars 1 forks source link

Fix/aws s3 collect multiple profiles #174

Closed Pokom closed 4 months ago

Pokom commented 4 months ago

Cherry picked a change from https://github.com/grafana/cloudcost-exporter/pull/169 to extend the configuration for AWS to have profiles which represents the profiles you want to pill data from.

Updated aws/s3 to become aware of multiple profiles and create a new costexplorer client per profile when fetching billing data. Updated s3.parseBillingData to return a slice of outputs so that we can merge them with other profiles before parsing out billing data.

Pokom commented 4 months ago

Thanks @inkel and @logyball for the reviews! Unfortunately this approach isn't going to work the way I had thought 😞 I have a strong preference not to create and maintain secrets to access multiple profiles, and the recommended route is to use roles and the stscreds Assume Role method. Yet Another Cloudwatch Exporter uses this technique to access multiple profiles from the same workload, but I still don't understand the underlying mechanism well enough.

As a temporary workaround, I've deployed an instance of cloudwatch-exporter to EKS clusters in each profile needed, and will follow pattern for the short term.