nerdswords / yet-another-cloudwatch-exporter

Prometheus exporter for AWS CloudWatch - Discovers services through AWS tags, gets CloudWatch metrics data and provides them as Prometheus metrics with AWS tags as labels
Apache License 2.0
953 stars 328 forks source link

Add support for AWS account aliases #1434

Closed thepalbi closed 2 months ago

thepalbi commented 2 months ago

AWS supports creating something called AWS account aliases. This are human friendly name that can be used to easily identify accounts. An account can have at most a single account alias (docs), and aliases are unique across an AWS network partition (docs).

This PR explores the possibility of pulling the AWS account alias, and if there's one, adding it as an account_alias to every metric, much like we add account_id.

Sample metric with acct alias

aws_lambda_throttles_sum{account_alias="palbi-personal",account_id="076251503",dimension_FunctionName="test-function",dimension_Resource="",name="arn:aws:lambda:us-east-2:076251503:function:test-function",region="us-east-2"}

TODO

thepalbi commented 2 months ago

@kgeckhart @cristiangreco have a few questions on how to best implement this:

  1. Overall, wdyt?
  2. I'd rather add this as an inline label to all metrics, rather than having an aws_account_info metrics, since it might be helpful to quickly look metrics by the alias, such as when using an account id
  3. Is it necessary to port this to the v1 clients?
  4. Also, should we gate this with a feature flag, or is it ok to attempt to fetch the alias, and if it fails log a warn and propagate just an "non-alias" (empty string).
kgeckhart commented 2 months ago
  1. 🔥 , be sure you document what permission is necessary for this to work
  2. I'm more apt to opt for an account info metric only because service info metrics are going to continue to grow the number of labels as we add more metadata sources. Since prom systems have a label limit it might be helpful to avoid chewing up two label places for account info everywhere.
  3. ATM v1 is still the default so I would say yes it's important to keep them in sync feature wise
  4. I think following the pattern of, context data on info metrics using a config flag is ideal vs a toggle
thepalbi commented 2 months ago

@kgeckhart as for 4), since this depends on IncludeContextOnInfoMetrics being enabled (since it consumes the contexts configured in each resources), and that's an opt-in flag; and at most it generates one active series per account, we could keep this under the same context. Otherwise, we could add a new config-wide (at the root level) config, to toggle if the user wants or not aws_account_info metrics.

Apart from that, the rest should be good.

Added the required perm to the suggested policy.

vainiusd commented 2 months ago

A possibility to add account_alias to all metrics would still be valuable. There is no means to have a single way to tag different namespace resources with some information and the account alias could be used to propagate such data without needing to do account_id to value mappings later on.

cristiangreco commented 2 months ago

I'm also tempted to say that this feature could be part of IncludeContextOnInfoMetrics. I see the point that it's just one new metric, but I guess that the account_alias info makes most sense next to the account_id rather than as a separate information.

thepalbi commented 2 months ago

I also prefer to include this as an extension of what IncludeContextOnInfoMetrics, inlining the account_alias label in all metrics, side by side with account_id. FYI @kgeckhart

thepalbi commented 2 months ago

@cristiangreco implemented all inlining the account_alias when available.