Add sustainability metrics and attributes to semantic conventions for hardware metrics

[bertysentry]

Use Case

We need to be able to report the carbon footprint of servers, network, storage, applications, and services. To allow that across the entire infrastructure, semantic conventions are required, starting with the underlying physical infrastructure.

Specifications

In addition to hw.power and hw.energy, add metrics to semantic conventions for hardware metrics, like:

• hw.abiotic_depletion_potential
• hw.product_carbon_footprint

Define a Site entity, i.e. a physical location with specific properties that can be measured with metrics like:

• hw.site.pue (more)
• hw.site.itue, hw.site.tue (more)
• hw.site.cue (more)
• hw.site.wue
• hw.site.ere, hw.site.erf (more)
• hw.site.ref, hw.site.oef (more)
• hw.site.cer (more)
• hw.site.electricity_cost
• hw.site.carbon_intensity

Additional context

More examples and links can be found in various places, but Green Software Foundation's Awesome Green Software is a good reference to start with, as well as its Hardware Efficiency page.

[bertysentry]

This needs to be moved to the semantic-conventions repository, please!

[nikimanoledaki]

cc @mkorbi @jpkrohling

[jpkrohling]

It looks like I can't move it, perhaps someone from the @open-telemetry/technical-committee can do it?

In any case: it looks like this needs a working group, and we might not have enough experts in the community to confidently drive this. I'd recommend gathering people who are passionate about this topic to collaborate as part of a WG.

A somewhat old document by @tedsuo describes how a new WG around semantic conventions can be formed: https://docs.google.com/document/d/1ghvajKaipiNZso3fDtyNxU7x1zx0_Eyd02OGpMGEpLE/edit . I'm not sure this was ever formalized in the full context of that doc, but I know the project management parts are described here: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/project-management.md

While I would really love to see progress on this important front, we might dedicate more attention and energy to stabilizing what we have already than accepting new conventions. This shouldn't stop you from making progress though.

[nikimanoledaki]

Agreed, bootstrapping a new WG is a lot of work and it creates a lot of overhead that requires time to be maintained. However I am not familiar with the process for this project specifically. @jpkrohling could you share next steps on how to create a WG for OTel?

Have we investigated if we can collaborate with the Green Software Foundation on the standards themselves? It would help to get guidance and make use of existing structures, groups, and processes. For example, the experience and expertise of the GSF Standards WG in creating standards for green software would be invaluable here. We could use a similar model to TAG ENV where the GSF helps with "what" standards to create and the TAG then helps with "how" to implement this with cloud-native tooling.

We could also try to help from TAG ENV - @mkorbi do you have ideas about how we could do that? The WG Green Reviews could benefit from standards such as this one in the long term. The infra we're setting up could also be used to trial these standards.

@bertysentry @mkorbi curious to hear what you think about this as members of the GSF and whether you could help raise this with the GSF?

[nikimanoledaki]

+cc @pyohannes

[pyohannes]

In any case: it looks like this needs a working group, and we might not have enough experts in the community to confidently drive this. I'd recommend gathering people who are passionate about this topic to collaborate as part of a WG.

A somewhat old document by @tedsuo describes how a new WG around semantic conventions can be formed: https://docs.google.com/document/d/1ghvajKaipiNZso3fDtyNxU7x1zx0_Eyd02OGpMGEpLE/edit . I'm not sure this was ever formalized in the full context of that doc, but I know the project management parts are described here: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/project-management.md

Some first steps I'd recommend:

• Gathering a group of experts who are willing to contribute to this.
• Writing a project tracking issue (see here and here for examples).
• Join the semantic conventions SIG meeting and talk about your intentions. This might be a first step to find possible sponsors from OTel side.

While I would really love to see progress on this important front, we might dedicate more attention and energy to stabilizing what we have already than accepting new conventions. This shouldn't stop you from making progress though.

This is true, if you're only interested in coming up with another area of experimental conventions, support might be limited. We'd appreciate somebody taking permanent ownership of this semantic convention area. Among other things, this involves creating prototypes, driving it to stability, and doing long-term maintenance.

[mkorbi]

We could highlight in a first-step this activity for both the TAG and the GSF to find further contributions and ideas. For GSF afaik there is no work in this direction. I dropped a msg to them. We might also check with SDIA if there is not something going on there. Also pinged.

On the other hand, I feel like this is starting in the middle. Almost all metrics require external input. Some could be added as labels, but then they don't need to be measured.

[nikimanoledaki]

Adding a reference to @musingvirtual's post in the TAG ENV Roadmap discussion:

I'm from the OpenTelemetry project. There seems to be interest in engaging us and I agree that telemetry is key to sustainability, but we need to have some dialogue and education between the two groups. It might also be good for someone to engage tag-observability as a whole. If you are further along with sustainability work in another project like Prometheus, we can possibly set up ways to ingest their signals as a quick option.

Sustainability is a question with a large cross-cutting scope and it's important that we actually have a thoughtful approach, it's not about one or two added metrics. To add sustainability to our spec is not just about defining a list of metrics, we'd need to make some decisions within the technical committee & the spec working group about where sustainability metrics would sit in our spec and also define how traces and logs and other signals would be handled, then there is project management to enable our working groups to implement the metrics once they are in spec, there is documentation, and finally we need to educate OpenTelemetry adopters so the metrics get used.

This is a lot of work for contributors who are already stretched thin and we need help from your group if we're going to implement it. In order to make this a priority we'd need to recruit and train some humans who can be dedicated to shepherding this along through the project.

Here are some specific ideas for collaboration:

My end user working group which organizes events can commit to hosting a speaker or panel from tag-sustainability if you provide them to frame the problem space. We can also commit to hosting ad-hoc follow up discussion sessions between TAG-Sustainability and OpenTelemetry until we decide whether to charter a working group or move forward in another way.

I can also commit to using our communication channels through the OpenTelemetry comms working group and asking observability experts to post about ongoing work, in order to try to locate people within the observability community who are interested in the problem.

Austin Parker and Ed Vielmetti are exploring a collaboration on how we might use existing OpenTelemetry to do sustainability work using the Equinix data center and Honeycomb, which will help us kick off education and awareness efforts with something we can implement right away.

OpenTelemetry has a lounge space on the KubeCon EU show floor and we can also do programming together in person there, and while the Observability Day program is being set right now we may be able to squeeze in a lightning talk or something.

Can TAG-Sustainability commit to hosting a talk from OpenTelemetry about the basics of how to collaborate with us and the basics of OpenTelemetry, so that you can help us resource this kind of work appropriately? And can your comms working group commit to using your channels to help us find contributors, and to cross-promote the discussions I'm outlining?

[bertysentry]

@trask This issue needs to be moved to the semantic conventions repo, please, thank you! 😊

[nikimanoledaki]

As per @jpkrohling's & @pyohannes's comments:

In any case: it looks like this needs a working group, and we might not have enough experts in the community to confidently drive this. I'd recommend gathering people who are passionate about this topic to collaborate as part of a WG.

A somewhat old document by @tedsuo describes how a new WG around semantic conventions can be formed: https://docs.google.com/document/d/1ghvajKaipiNZso3fDtyNxU7x1zx0_Eyd02OGpMGEpLE/edit . I'm not sure this was ever formalized in the full context of that doc, but I know the project management parts are described here: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/project-management.md

• Writing a project tracking issue (see here and here for examples).
• Join the semantic conventions SIG meeting and talk about your intentions. This might be a first step to find possible sponsors from OTel side.

Would anyone like to volunteer to kickoff the WG creation process by filing a Project Tracking Issue, please?

[Spazzy757]

Hi,

Ive startyed this issue: https://github.com/open-telemetry/community/issues/2020

Still fleshing out the details, but happy to push this. Im also getting access to GSF slack to see if we can get some input there

[danielgblanco]

Considering we have the project aforementioned and the following project board

https://github.com/orgs/open-telemetry/projects/93/views/1

Does it still make sense to keep this issue active?

For anyone else seeing this, please follow the info in https://github.com/open-telemetry/community?tab=readme-ov-file#special-interest-groups if you'd like to participate in this SIG!

[bertysentry]

We can close this issue. Thank you all for the interest and for the work that will be done to achieve this! 😊