[ACTION] Proposal 3 - Report

[ADiTuri]

Task Description

Parent Issue https://github.com/cncf-tags/green-reviews-tooling/issues/83

This issue is about discussing a strategy on the reporting capabilities of a Green Review. There are two open points to address:

• Metrics: What kind of metrics do we select for a green review?
• Long term storage: Where and how do we save the green metrics?

Metrics

At the moment we have agreed on the SCI score calculation, details of how we compute the SCI score can be found here.

Open points on the current approach

From the last milestone we had two open points:

1. Embodied carbon: plays a significant role in the SCI. This might not be relevant for the CNCF projects interested in the review, cause it is just dependent on the infrastructure we choose to use.
2. Functional unit: We choose a time functional unit of 15 minutes, are we fine with this or is there any other improvement we would like to do?

Possible extensions

This issue would also like to address possible extensions on the metrics we would like to consider:

• Traditional SRE metrics (provider could be the simple k8s SDK)
• More low level energy metrics (right now we have an aggregate at container level)
• Any other metric that might be interesting for future release

Long Term storage

This section works under the assumption that we have already a chosen set of green metrics. At the moment we don't store any metrics, the energy metric is just show live on the dashboard for the target container. We would like to store the metrics for future analytical purposes. The proposal should suggest:

• a long term storage technology
• a process for storing the green metrics
• integration with the current grafana dashboard

Goals to achieve

• [ ] Draft the proposal for the metrics extension
• [ ] Draft the proposal for the long term storage
• [ ] Review the proposals and merge

Nice to have

• [ ] Find experts for the metric topic
• [ ] FInd experts for the long term storage topic

[rossf7]

The 3 SRE metrics for CPU and memory requested by the Falco team are in this comment

We should consider these in the proposal to not lose the previous discussion.

[ChrisChinchilla]

@rossf7 and @ADiTuri I am more than happy to think about the metrics part to begin with as it fits with my work right now too. I am less confident or experienced on ideas for the storage part, but it's also an excuse to explore some new database engines…

[rossf7]

@nikimanoledaki @ChrisChinchilla in the last meeting I said I'd look into long term storage options. Here is the write up. I'll present it at the meeting tomorrow.

We can also discuss then how best to integrate it into the proposal PR. Posting it here so its somewhere public. Sorry for the wall of text!

Git

Store results in JSON or Markdown with the file being committed by the GitHub Actions workflow.

Strengths

• Simple and no infra to manage.
• Results are publicly accessible.
• Results can be migrated to more complex storage option later.

Weaknesses

• Static data.
• Not interactive so no ability to compare results.

Object Storage (S3 compatible)

Upload results to object storage from GitHub Actions workflow.

Strengths

• Simple.
• External backup outside of the cluster.

Weaknesses

• Data is private and not publicly accessible.
• Not interactive so no ability to compare results.

3 hosting options.

AWS S3

Store results in an S3 bucket.

Strengths

• Simple and no infra to manage.
• We already have S3 bucket in CNCF AWS account for storing cluster OpenTofu state files.

Weaknesses

• Proprietary cloud service.
• Need to check we can use existing bucket for storing more data.

MinIO

Run Equinix Metal node running MinIO.

Strengths

• S3 compatible but not a proprietary service.
• Open Source (AGPL license).

Weaknesses

• Involves running an additional server.
• Still needs external backup in case server fails.

Apache Ozone

Run Equinix Metal node running Apache Ozone.

Strengths

• Same as MinIO.
• Open Source (Apache 2 license).

Weaknesses

• Same as MinIO.

In Cluster Storage with external backup

Current cluster runs K3s which can be configured to use local storage. We still need an external backup but we could use object storage for this.

Strengths

• Cluster already has Prometheus and Grafana deployed.
• Dynamic queries like comparing results can be implemented.

Weaknesses

• We still need an external backup of the results.
• Needs a cluster with at least one node running 24/7.

[nikimanoledaki]

Hey @rossf7! 👋 When you have time, could you add the strengths and weaknesses of using the devstats Postgres DB that we discussed in the last meet, please? Thank you!

• Postgres database used by DevStats

I realise you are probably drafting and incorporating the feedback into Proposal 3 at the moment. Feel free to add it directly there rather than in this issue ☺️ Thank you for summarising everything!! 💯