Request access to AWS billing data

[colinbruce]

Service name

moj-crimebillingonline

Service environment

• [ ] Dev / Development
• [ ] Staging
• [x] Prod / Production
• [ ] Other

Impact on the service

Reducing time taken to manually compile billing data

Problem description

Currently, to compile the quarterly performance platform data, a single user in our project logs in to the AWS master account, accesses the billing reports filters the accounts to show moj-crimebillingonline and copies the values for the 3 months of the quarter in question. These are then manually converted to from USD to GBP , totalled, combined with a count of transactions in the system, copied to a spreadsheet and uploaded to the GDS performance platform to generate this...

I'm looking for a method of accessing the billing data so this entire process can be automated and run from the service direct via a cron job

Happy to discuss solutions e.g. using lambda, an IAM account with necessary permissions to request data, etc..

Contact person

Colin Bruce, @colinbruce on slack

Severity

Low, an intermediate step of manually entering the amounts has been undertaken but full automation removes the need to have lots of users manually extracting this data.

[jasonBirchall]

@AntonyBishop I think this needs to go into the backlog grooming.

[AntonyBishop]

Requested additional info from Steve Marshall

[sid-secops]

We would need an IAM user account with restricted permissions (read only) for API access to the AWS Billing and Cost Management. The IAM user account should have the below policy, which will provide access to Billing and Cost Management, AWS usage Reports, AWS Cost and Usage Report

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "aws-portal:ViewUsage",
                "aws-portal:ViewBilling",
                "ce:Get*"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Deny",
            "Action": [
                "aws-portal:*Account",
                "aws-portal:*Budget",
                "aws-portal:ModifyBilling",
                "aws-portal:*Payment*",
                "iam:*"
            ],
            "Resource": "*"
        }
    ]
} 

[sid-secops]

To test the policy by attaching to an IAM user in the AWS test account. once the script works as expected, replicate the IAM user and policy in the AWS master account. To discuss with team, Steve and update Colin

[sid-secops]

the iam user has been created in the test aws account. credentials have been shared with Colin. He should be able to work out the script, test and get back to us

[colinbruce]

@kcbotsh can this be re-opened? I have access on a non-live account, but access to the actual billing data I need on the moj-dsd AWS account has not yet been provided. The permissions provided above were a test to see if the AWS CLI could provide the necessary data.

[sid-secops]

@colinbruce please could you let me know if the permissions were sufficient for the task. I can create a new iam user in the prod account and let you know

[colinbruce]

The permissions you set up were fine @sid234. What's the policy on team credentials? I'm keen not to become another blocker in the event I leave and my account is deleted. Can it be set up in the applications name? cccd or claim-crown-court-defence? There is a google group email that can be assigned if that helps? crowncourtdefence@digital.justice.gov.uk

[sid-secops]

thanks Colin, I will create claim-crown-court-defence user in the prod account and add the permissions

[sid-secops]

user created and shared details