💸 Enable AWS Split Cost Allocation Data for Amazon EKS

[jacobwoffenden]

User Story

As a Modernisation Platform customer that runs EKS I want cost allocation data for my Kubernetes workloads So that I can attribute it back to users of Analytical Platform's compute cluster

Value / Purpose

Analytical Platform's EKS cluster runs varied workloads (Analytical Platform's CDE, Airflow, GitHub Actions runners) and it would be useful to attribute the cost against them, not (currently at least) for charging back, just to make visible.

Useful Contacts

@jacobwoffenden

Additional Information

We can run this in cluster with Kubecost, but I would like to enable AWS native so we don't have to run another service for it https://aws.amazon.com/blogs/aws-cloud-financial-management/improve-cost-visibility-of-amazon-eks-with-aws-split-cost-allocation-data/

Definition of Done

• [ ] New terraform resource implemented in aws-root-account
• [ ] Customer confirms reports have information that they require

[dms1981]

I'll go back to @jacobwoffenden to refine this one a bit better - at present there's no DoD, and the requirement is tied to a single user of EKS on MP - in general, anyone using K8s is directed towards Cloud Platform.

[dms1981]

If this is as simple as enabling Split cost allocation data in certain member accounts, this might be an easier ticket than it first appears, but a definition of done would help us understand better.

[dms1981]

After doing some more reading I think it can be enabled through this terraform resource, but I think it may need some broader consideration as it has to be implemented in us-east-1.

Also this looks like it will need to be implemented through the MOJ Master Account (AWS Organizational root):

NOTE: If AWS Organizations is enabled, only the master account can use this resource.

[SimonPPledger]

At request of AP, moving in to sprint

[dms1981]

Looking at the relevant resource there are a few attributes we'd need to be clearer on:

  additional_schema_elements = ["RESOURCES", "SPLIT_COST_ALLOCATION_DATA"]
  s3_bucket                  = "example-bucket-name"
  additional_artifacts       = ["REDSHIFT", "QUICKSIGHT"]

Are the default additional_schema_elements sufficient?

Is there a predefined s3_bucket for these reports? If one needs to be created, how should it be secured (eg, against users? roles? keys in the bucket?

What additional_artificats - if any - are required?

[dms1981]

Some investigation shows that, actually, this can't be managed in code. I've reached out to the root-account-team for a steer.

[dms1981]

Change in the root account to enable trusted access is blocked by some secrets that were marked for deletion.

[dms1981]

Resolved! Thanks to @julialawrence & @davidkelliott !