Open trent-s opened 2 months ago
Slightly separate point. The SusQL operator does need a better operator level configuration mechanism.
Implemented a prototype version that uses a configmap called susql-config
if it is in the current namespace. (Sample file at doc/susql-config.yaml
) Otherwise default values built into main.go
for OperatorHub deployment, and deploy.sh
& values.yaml
for Helm deployment. Can improve documentation. Also, would be nice to update the Helm chart to reference the ConfigMap...
Phase 1 works! suqltop
shows totalCarbon
data from labelgroup. Likewise when operator is deployed from operatorhub.io data can be graphed in the Observe->metric tab by searching for susql_total_carbon_dioxide_grams
.
Phase 2 is looking good! susqltop
and the Observe->Metric tab provide CO2 information, and the carbon intensity is updated periodically according to the ConfigMap parameters! Program logic seems solid...
Carbon footprint is a useful metric to output. Tools such as the following coupled with location data associated with nodes could be useful: https://github.com/Green-Software-Foundation/carbon-aware-sdk
High Level Plan:
[x] Phase 1 - Simple static CO2 Calculation (AKA "Getting our feet wet") Goal: demonstrate that SusQL can report usable carbon dioxide emissions aggregation data as PoC
Allow user defined energy->carbon conversion factor (via optional operator wide ConfigMap)
CARBON-INTENSITY
-- Carbon Intensity Factor in grams CO2 / Joule (example indoc/susql-config.yaml
)update
susqltop
to show CO2 datainclude ongoing storage of CO2 emission data in parallel with energy storage
totalCarbon
(in units of grams CO2).susql_total_carbon_dioxide_grams
update any tools, etc to correctly handle/display this new CO2 data
[x] Phase 2 - Ongoing conversion from single provider (AKA "Wading deeper") Goal: demonstrate that SusQL can easily use current regional carbon intensity data. As of this writing, this method is largely finished, though testing and verification is still under progress.
Add the following parameters to operator level ConfigMap to enable querying carbon intensity data in real time from a free data provider:
CARBON-METHOD
- Either "static", "simpledynamic", or "sdk" (default to "static")CARBON-INTENSITY-URL
- URL to query carbon intensity dataCARBON-LOCATION
- Location identifier for carbon intensity query. (Depends on carbon intensity data provider.)CARBON-QUERY-RATE
- How often to query carbon intensity. (Default to 7200 seconds)CARBON-QUERY-FILTER
- A parameter togojq
to obtain desired value. (e.g.,carbonIntensity
)CARBON-QUERY-CONV-2J
- A conversion factor as necessary to convert from provider units to grams of CO2 per Joule.Calculate total carbon emission data the same as TotalEnergy. (Aggregate total in LabelGroup. Time series data in Prometheus.)
[ ] Phase 3 - Full carbon-aware-sdk utilization (AKA "Dive in all the way") Goal: demonstrate integration with industry standard carbon aware toolkit
Allow user customization of carbon data subscription provider
Gather data from carbon-aware-sdk rather than directly from provider
Ensure full operator level customization and integration.
Define and document priorities. e.g., user can choose between phases 1, 2, & 3 carbon calculation
Initially, should be able to allow "static" carbon intensity setting within the sdk for PoC purposes.
[ ] Additionally
It would be nice to automatically restart the operator if a change is detected in the ConfigMap
Enhance Helm chart deploy flow to reference ConfigMap somehow
Speaking of the ConfigMap, update documentation!
Take care of capitalization: gram, carbon, etc are only capitalized according to general capitalization rules.
The word "carbon" is often used as casual shorthand for "carbon dioxide". Take care to use correctly.
LabelGroup and ConfigMap should always be written in UpperCamelCase.