Open choldgraf opened 1 year ago
I think that this is a good compromise, because we (1) tie cloud bills for each hub to a reasonable estimate of their usage, and (2) entirely cover our cloud costs.
I agree, I think this is a good enough compromise for now as a first strategy that we can evolve/improve in the future with iterative steps.
Current process is using per namespace
Use Grafana Query:
sum(
kube_pod_container_resource_requests{resource="memory"}
) by (namespace)
Step: 24h0m0s Set TZ to UTC, Use date selector. Fiscal quarter queries also work.
This manual process is being replaced/streamlined but for context (will also be added to docs)
=EPOCHTODATE(A2,2)
=SUMPRODUCT(Daily!D$2:D$272, MONTH(Daily!$B$2:$B$272)=MONTH($A2))
ranges will need to be adjusted per Daily row=SUM(C2:W2)
=DIVIDE(Monthly!C2*100, Monthly!$B2)
='Shared Usage'!B2 * $B2 / 100 + ('Shared Usage'!B2 * ($V2 + $W2)) / 100
Context
Here's a proposal for how to decide on the cloud costs for shared hubs on Google Cloud[^1].
[^1]: - For a proposal on communities with dedicated clusters, see https://github.com/2i2c-org/team-compass/issues/498
Overview
Calculating cloud costs on a shared cluster is difficult because we do not get an exact breakdown of costs incurred by community hub. Instead we have information about where our costs came from (machines, storage, etc) as well as our own information about hub usage over a period of time from Grafana.
To that extent, we will calculate hub costs by this process:
(hub_RAM_used / total_cluster_RAM_used) * cloud_bill_for_cluster
This assumes that RAM is the largest driving factor of cloud costs, which is reasonable for most communities. It will break down if a community uses a lot of storage (in that case, some hubs would be charged more than we want, and others less), but this stop-gap billing strategy can intentionally not address that use-case so that we can iterate on this in the future. I think that this is a good compromise, because we (1) tie cloud bills for each hub to a reasonable estimate of their usage, and (2) entirely cover our cloud costs.
Instructions to do this
First, figure out the total cloud cost that we incurred on the shared cluster, called
two-eye-two-see
.Next, figure out the RAM used by each hub over the same month.
TODO: We need to figure out how to calculate RAM usage by community over time.
To do
modified from: https://github.com/2i2c-org/infrastructure/issues/730#issuecomment-978291582