monitoringartist / grafana-aws-cloudwatch-dashboards

:cloud: 40+ Grafana dashboards for AWS CloudWatch metrics: EC2, Lambda, S3, ELB, EMR, EBS, SNS, SES, SQS, RDS, EFS, ElastiCache, Billing, API Gateway, VPN, Step Functions, Route 53, CodeBuild, ...
https://grafana.com/orgs/monitoringartist/dashboards
GNU General Public License v3.0
2.05k stars 790 forks source link

AWS Billing for Linked account #91

Closed nbari closed 3 years ago

nbari commented 3 years ago

It is possible to get billing data when using credentials only from a Linked account?

I created a user with these permissions: CloudWatchReadOnlyAccess, ResourceGroupsandTagEditorReadOnlyAccess, AmazonEC2ReadOnlyAccess

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "autoscaling:Describe*",
                "cloudwatch:Describe*",
                "cloudwatch:Get*",
                "cloudwatch:List*",
                "logs:Get*",
                "logs:List*",
                "logs:StartQuery",
                "logs:StopQuery",
                "logs:Describe*",
                "logs:TestMetricFilter",
                "logs:FilterLogEvents",
                "sns:Get*",
                "sns:List*"
            ],
            "Effect": "Allow",
            "Resource": "*"
        }
    ]
}

I can get metrics but for some reason, the AWS/Billing is always empty, I just updated to https://github.com/monitoringartist/grafana-aws-cloudwatch-dashboards/blob/master/aws-billing/aws-billing.json but still can't fetch any data.

The AWS/Billing works for Linked accounts ? or is only available from the main account?

Screenshot 2021-02-27 at 21 07 03
jangaraj commented 3 years ago

I don't have idea (I work with cross account CloudWatch IAM access usually). If you can get linked accounts billing details with aws cli from the main account, then it should be possible.

IMHO you have wrong expectations - you have removed all dimensions and you are expecting to see "all" data now. But it doesn't work like that in the CloudWatch. There is LinkedAccount dimension, so I use it. Please contact your AWS support if you have a problem with that. It isn't in our scope to support dashboard with your own modifications. I gave you just clue with dimensions usage.

nbari commented 3 years ago

Hi, thanks for the hint If you can get linked accounts billing details with aws cli from the main account, then it should be possible.

just in case, I fixed the problem by creating this policy and attaching it to the metrics user:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "VisualEditor0",
      "Effect": "Allow",
      "Action": [
        "ce:DescribeCostCategoryDefinition",
        "ce:GetRightsizingRecommendation",
        "ce:GetCostAndUsage",
        "ce:GetSavingsPlansUtilization",
        "ce:GetAnomalies",
        "ce:GetReservationPurchaseRecommendation",
        "ce:ListCostCategoryDefinitions",
        "ce:GetCostForecast",
        "ce:GetPreferences",
        "ce:GetReservationUtilization",
        "ce:GetCostCategories",
        "ce:GetSavingsPlansPurchaseRecommendation",
        "ce:GetDimensionValues",
        "ce:GetSavingsPlansUtilizationDetails",
        "ce:GetAnomalySubscriptions",
        "ce:GetCostAndUsageWithResources",
        "ce:DescribeReport",
        "ce:GetReservationCoverage",
        "ce:GetSavingsPlansCoverage",
        "ce:GetAnomalyMonitors",
        "ce:DescribeNotificationSubscription",
        "ce:GetTags",
        "ce:GetUsageForecast"
      ],
      "Resource": "*"
    }
  ]
}

Also just found that the region must be us-east-1

jangaraj commented 3 years ago

This is weird. Cost explorer permissions (ce:*) give you access to CloudWatch billing metrics for linked accounts. Grafana is not able to query cost explorer data (https://github.com/grafana/grafana/issues/9953), so I don't understand.

nbari commented 3 years ago

Hi @jangaraj you are correct the ce:* only helped me to get metrics from the cli using aws ce get-cost-and-usage ... the script that helped me to get/verify keys were working using cloudwatch is:

aws --region us-east-1 cloudwatch get-metric-statistics \
    --namespace "AWS/Billing" \
    --metric-name "EstimatedCharges" \
    --dimension "Name=Currency,Value=USD" \
    --start-time $(date -v -12H '+%Y-%m-%dT%H:%M:%SZ') \
    --end-time $(date '+%Y-%m-%dT%H:%M:%SZ') \
    --statistic Maximum \
    --period 60 \
    --output text | sort -r -k 3 | head -n 1 | cut -f 2

I had to use us-east-1 and also I was misled since I was testing at the end/beginning of the month and metrics were returning 0 (period 30 days)