Carbon Calculation

[trent-s]

Carbon footprint is a useful metric to output. Tools such as the following coupled with location data associated with nodes could be useful: https://github.com/Green-Software-Foundation/carbon-aware-sdk

High Level Plan:

• [x] Phase 1 - Simple static CO2 Calculation (AKA "Getting our feet wet") Goal: demonstrate that SusQL can report usable carbon dioxide emissions aggregation data as PoC

• Allow user defined energy->carbon conversion factor (via optional operator wide ConfigMap)

  • CARBON-INTENSITY-- Carbon Intensity Factor in grams CO2 / Joule (example in doc/susql-config.yaml)
  • Grams CO2 used since it matches the Carbon Aware SDK. Joules used since SusQL uses this internally
  • Default value of 0.00011583333 grams CO2/Joule based on eGrid data for USA in 2021. Ref: https://www.epa.gov/energy/greenhouse-gases-equivalencies-calculator-calculations-and-references

• update susqltop to show CO2 data

• include ongoing storage of CO2 emission data in parallel with energy storage

  • Tracking in LabelGroup as totalCarbon (in units of grams CO2).
  • Tracking in Prometheus as susql_total_carbon_dioxide_grams

• update any tools, etc to correctly handle/display this new CO2 data

• [x] Phase 2 - Ongoing conversion from single provider (AKA "Wading deeper") Goal: demonstrate that SusQL can easily use current regional carbon intensity data. As of this writing, this method is largely finished, though testing and verification is still under progress.

• Add the following parameters to operator level ConfigMap to enable querying carbon intensity data in real time from a free data provider:

  • CARBON-METHOD - Either "static", "simpledynamic", or "sdk" (default to "static")
  • CARBON-INTENSITY-URL - URL to query carbon intensity data
  • CARBON-LOCATION - Location identifier for carbon intensity query. (Depends on carbon intensity data provider.)
  • CARBON-QUERY-RATE - How often to query carbon intensity. (Default to 7200 seconds)
  • CARBON-QUERY-FILTER - A parameter to gojq to obtain desired value. (e.g., carbonIntensity)
  • CARBON-QUERY-CONV-2J - A conversion factor as necessary to convert from provider units to grams of CO2 per Joule.
  • (The default values are verified to work as of this writing, and are designed to be sufficiently flexible to support other providers. Please open an issue if other tunable options would useful.)

• Calculate total carbon emission data the same as TotalEnergy. (Aggregate total in LabelGroup. Time series data in Prometheus.)

• [ ] Phase 3 - Full carbon-aware-sdk utilization (AKA "Dive in all the way") Goal: demonstrate integration with industry standard carbon aware toolkit

• Allow user customization of carbon data subscription provider

• Gather data from carbon-aware-sdk rather than directly from provider

• Ensure full operator level customization and integration.

• Define and document priorities. e.g., user can choose between phases 1, 2, & 3 carbon calculation

• Initially, should be able to allow "static" carbon intensity setting within the sdk for PoC purposes.

• [ ] Additionally

• It would be nice to automatically restart the operator if a change is detected in the ConfigMap

• Enhance Helm chart deploy flow to reference ConfigMap somehow

• Speaking of the ConfigMap, update documentation!

• Take care of capitalization: gram, carbon, etc are only capitalized according to general capitalization rules.

• The word "carbon" is often used as casual shorthand for "carbon dioxide". Take care to use correctly.

• LabelGroup and ConfigMap should always be written in UpperCamelCase.

[trent-s]

Slightly separate point. The SusQL operator does need a better operator level configuration mechanism.

Implemented a prototype version that uses a configmap called susql-config if it is in the current namespace. (Sample file at doc/susql-config.yaml) Otherwise default values built into main.go for OperatorHub deployment, and deploy.sh & values.yaml for Helm deployment. Can improve documentation. Also, would be nice to update the Helm chart to reference the ConfigMap...

[trent-s]

Phase 1 works! suqltop shows totalCarbon data from labelgroup. Likewise when operator is deployed from operatorhub.io data can be graphed in the Observe->metric tab by searching for susql_total_carbon_dioxide_grams.

[trent-s]

Phase 2 is looking good! susqltop and the Observe->Metric tab provide CO2 information, and the carbon intensity is updated periodically according to the ConfigMap parameters! Program logic seems solid...