search

2i2c-org / team-compass

Organizational strategy, structure, policy, and practices across 2i2c.
https://compass.2i2c.org
4 stars 13 forks source link

Calculate cloud costs for shared hubs on a cluster on Google Cloud #499

Open choldgraf opened 1 year ago

choldgraf commented 1 year ago

Context

Here's a proposal for how to decide on the cloud costs for shared hubs on Google Cloud[^1].

[^1]: - For a proposal on communities with dedicated clusters, see https://github.com/2i2c-org/team-compass/issues/498

Overview

Calculating cloud costs on a shared cluster is difficult because we do not get an exact breakdown of costs incurred by community hub. Instead we have information about where our costs came from (machines, storage, etc) as well as our own information about hub usage over a period of time from Grafana.

To that extent, we will calculate hub costs by this process:

This assumes that RAM is the largest driving factor of cloud costs, which is reasonable for most communities. It will break down if a community uses a lot of storage (in that case, some hubs would be charged more than we want, and others less), but this stop-gap billing strategy can intentionally not address that use-case so that we can iterate on this in the future. I think that this is a good compromise, because we (1) tie cloud bills for each hub to a reasonable estimate of their usage, and (2) entirely cover our cloud costs.

Instructions to do this

First, figure out the total cloud cost that we incurred on the shared cluster, called two-eye-two-see.

Next, figure out the RAM used by each hub over the same month.

TODO: We need to figure out how to calculate RAM usage by community over time.

To do

modified from: https://github.com/2i2c-org/infrastructure/issues/730#issuecomment-978291582

damianavila commented 1 year ago

I think that this is a good compromise, because we (1) tie cloud bills for each hub to a reasonable estimate of their usage, and (2) entirely cover our cloud costs.

I agree, I think this is a good enough compromise for now as a first strategy that we can evolve/improve in the future with iterative steps.

pnasrat commented 1 year ago

Current process is using per namespace

Use Grafana Query:

sum(
  kube_pod_container_resource_requests{resource="memory"}
) by (namespace)

Step: 24h0m0s Set TZ to UTC, Use date selector. Fiscal quarter queries also work.

This manual process is being replaced/streamlined but for context (will also be added to docs)

  1. Manually downloaded the .csv from Grafana of above query (using Data options: Series joined by time, Formatted data) and then imported into google sheets in a Daily sheet.
  2. Add 2nd column for date from timestamp =EPOCHTODATE(A2,2)
  3. Clean data by replacing all undefined values with zero and removing any extra columns (eg Namespace and Value) and the last row if it is in the subsequent month
  4. Create a monthly tab with first row column headings Month, Total RAM, HUBS... from Daily sheet
  5. Populate the Month column with the months of the range (eg 1/22.. 2/22)
  6. In the first Sum daily into per month by namespace totals per hub for all hubs and months =SUMPRODUCT(Daily!D$2:D$272, MONTH(Daily!$B$2:$B$272)=MONTH($A2)) ranges will need to be adjusted per Daily row
  7. For the Total RAM column sum the remaining columns in row for each month eg =SUM(C2:W2)
  8. Create a shared usage sheet with first column Month and remaining columns HUBS ..., add rows for each month in Monthly sheet
  9. Using a percentage (could be a ratio) calculate shared usage for each hub/month =DIVIDE(Monthly!C2*100, Monthly!$B2)
  10. Add a shared cost sheet, columns Month, Total Cost, HUBS ...
  11. Calculate per hub cost per month including support infrastructure based on usage for kube-system and support eg if kube-system and support are in V and W columns ='Shared Usage'!B2 * $B2 / 100 + ('Shared Usage'!B2 * ($V2 + $W2)) / 100