Open clalbus opened 1 year ago
At a high level this makes sense. I have never worked with monitoring accounts, can you describe a little more how you would build the overall setup? How would you run the exporter (a single exporter for the whole account?) How would you collect, and at all the stages distinguish, the metrics from the different accounts? The account ID is not very memorable, how could we make it so that the metrics are labeled with something nicer when it's all done?
At a high level this makes sense. I have never worked with monitoring accounts, can you describe a little more how you would build the overall setup? How would you run the exporter (a single exporter for the whole account?) How would you collect, and at all the stages distinguish, the metrics from the different accounts? The account ID is not very memorable, how could we make it so that the metrics are labeled with something nicer when it's all done?
We have the same desired use case as @clalbus — it aligns with suggested AWS best practices for AWS Organizations. AWS CloudWatch monitoring accounts is one example of the many supported solutions that follow the delegated admin account pattern
For a cloudwatch_exporter
integration with CloudWatch monitoring accounts, a high level overview of the setup could be:
Deploy cloudwatch_exporter
in the monitoring account with the following proposed configuration (based off of the suggestion made by @clalbus):
---
region: eu-west-1
metrics:
- aws_namespace: AWS/ELB
+ aws_account_id: 111111111111
+ aws_account_label: dev
aws_metric_name: RequestCount
aws_dimensions: [AvailabilityZone, LoadBalancerName]
aws_dimension_select:
LoadBalancerName: [myLB]
aws_statistics: [Sum]
- aws_namespace: AWS/ELB
+ aws_account_id: 222222222222
+ aws_account_label: qa
aws_metric_name: RequestCount
aws_dimensions: [AvailabilityZone, LoadBalancerName]
aws_dimension_select:
LoadBalancerName: [myLB]
aws_statistics: [Sum]
The aws_account_label
would add an additional aws_account
label to the metrics. I thought of suggesting that cloudwatch_exporter
would query AWS organizations and determine this label automatically, but it gives less control to the user, is probably a bad idea because we just want cloudwatch_exporter
to do one thing and do it well, and it wouldn't work with CloudWatch sharing accounts that exist outside the AWS organization of the monitoring account.
Without this feature, you would not be able to tell which account the metrics came from.
But right now monitoring accounts are not supported at all, because we would need to add IncludeLinkedAccounts to the ListMetrics call.
One thing I have not fully considered yet is how a configuration would look like in a multi account, multi region setup, which is supported by CloudWatch. I suppose you would have a configuration per region, though admittedly that's pretty repetitive.
Sounds good, I think this is a good feature and a clean design. One thing I would add is that if aws_account_label
is not specified, we use the account ID. This way the metrics are still unique, and the label setting becomes a convenience not a necessity.
I have a similar use case where I want to collect the metrics from a different AWS Account into our main account's Prometheus without the need to deploy an additional prometheus-cloudwatch-exporter instance that would assume the cross-account IAM role. Looking forward to monitoring-account support in prometheus-cloudwatch-exporter.
I have a similar use case and i have 2 option possible for it :
option 1 : using role_arn per metric like to be able to pull metrics from differents account
- aws_namespace: AWS/CertificateManager
aws_metric_name: DaysToExpiry
aws_dimensions: [CertificateArn]
aws_statistics: [Maximum]
- aws_namespace: AWS/CertificateManager
aws_metric_name: DaysToExpiry
aws_dimensions: [CertificateArn]
aws_statistics: [Maximum]
role_arn: arn:aws:iam::123456789:role/CloudWatchCrossAccountRole
option 2 : Monitoring account support
any updates about it ?
Use case. Why is this important?
In multi account setup it is suggested to use monitoring accounts for easier metric access i.e. view multi account metrics through Cloudwatch in single account.
It is possible to filter metrics by account ID using
setAccountId(String accountId)
on MetricDataQuery when retrieving Cloudwatch metrics from monitoring account.However, the above applies only to users who use monitoring accounts.
WHY
We are picking up pace with multi account setup and would like to simplify Cloudwatch metric ingestion into Prometheus via monitoring account.
Not only it simplifies IAM management i.e. no more role/policy per account, but with account filtering it allows us to re-use existing Cloudwatch exporters (requiring simple config change).
How do you think the new configuration should look like?
Example: