UC: Use of Offsets over Efficiency

[jawache]

The money spent purchasing market-based instruments (offsets) means investment goes away from green software and energy efficiency initiatives and towards low-carbon energy suppliers and carbon offset projects. If our goal is to incentivize investment into making software more efficient, this runs counter to our goals.

• A label of A needs to celebrate the work the organization has done to build efficient software to encourage more efficient software to be built. If a label of A was given instead to the organization that bought the most offsets, then this is incentivizing the purchasing of offsets.

Equivalence

• The equivalent for nutri-score would be a food product that is very unhealthy for you scoring an A because the organization invested in a program that gave some other people, in some other place, healthy food.
• The equivalent for energy star would be a washing machine that scored an energy star but was incredibly inefficient in real-life, the organization just bough some energy offsets to counter your energy consumption rather than actually engineer a more efficient washing machine.

Effects

• If you can buy your way to an A, this erodes public trust in a label.
• If a product that obviously emits more emissions but scores a better rating because of offset purchases than a product known to be incredibly energy efficient, this will again erode the trust in the label.
• Might encourage the purchase of very low-quality (cheap) offsets with limited evidence of additionality. Offsets have a wide quality rating with very limited standards and consensus; you can pay $1 to offset a tonne by planting a tree or $1000 to offset with Direct Air Capture. You can offset with unbundled renewable energy for a fraction of the price of bundled renewable energy.
• If one organization starts buying offsets to improve its rating, this forces everyone else to buy offsets. Just like if one person starts doping in the Olympics and starts winning, everyone has to start doping to compete.
• You can offset to 0 emissions, and be carbon neutral, which means the score is 0. If the highest score possible is 0 and the majority of products who rate themselves score 0, then a percentile rating system fundamentally fails.

Counter

• Align fully with the SCI, SCER is the label/rating-system/disclosure-mechanism just for SCI scores. SCI already has clear measures in place to exclude market-based instruments.

[chrisxie-fw]

1. I really enjoyed your example about the washing machine. In real life, I once bought a "Water Efficient" HE Washing Machine. I'm guessing that, in the pursuit of water efficiency, the manufacturer designed it to use so little water that the clothes weren't even wet at the end of the wash cycle. I'm curious if others have had similar experiences. This resonates with your point about prioritizing one aspect, like efficiency or "offsets", while sacrificing performance or real CO2 reduction. Thus, I agree that SCI could be the ultimate or default measurement in the base SCER specification.

2. However, to make it easier for the industry to adopt SCER, it might be better to leave the decision to use SCI or not to the implementers of the SCER framework, at least in the initial stages or earlier versions of the SCER standard. This way, we balance the need for adoption with the ultimate goal of CO2 efficiency.

3. In addition, regulators are working to reestablish trust in carbon markets, as mentioned in this post. Let them do their thing while we focus on the bare minimums that the community/industry can agree on.

SCER is similar to ISO quality standards in its framework and processes. For example, ISO 9001, an international standard for Quality Management Systems (QMS), includes key components such as Organization Context, Leadership, Planning, Operation, Performance Evaluation, and Improvement. Each organization may implement these standards differently; some might have the CEO lead the implementation, while others might choose the COO. ISO 9001 does not mandate who must lead, as long as the organization follows the standard processes to achieve certification.

In the context of SCER, as long as an organization adheres to the basic four steps in the SCER standard framework, they are considered SCER-compliant. In future versions of SCER, if we can bring major stakeholders together to establish standard categories, benchmarking workloads, rating systems, and labeling, we could have a concrete implementation of the base SCER standard. This would allow the industry to measure the CO2 efficiency of Large Language Models (LLMs) across different vendors, whether open/source or proprietary.

Alternatively, the Green Software Foundation (GSF) could create a SCER for LLMs CO2 Efficiency Rating lab, defining the standard hardware, software, workload, rating system, and labels, and publish the results on the GSF website. This approach is included in the SCER WG's roadmap, as described in the Scope and Objectives section of the SCER Charter regarding "Creating a Green Software Certification Lab (GSCL)." This is also available in SCER WG's Readme on Creating a Green Software Certification Lab (GSCL).