search

Green-Software-Foundation / scer

Software Carbon Efficiency Rating
Other
29 stars 5 forks source link

2024.04.03 #44

Closed seanmcilroy29 closed 6 months ago

seanmcilroy29 commented 7 months ago

2024.04.03 Agenda/Minutes

Time: Bi-weekly @ 1700 (GMT) - See the time in your timezone

Antitrust Policy

Joint Development Foundation meetings may involve participation by industry competitors, and the Joint Development Foundation intends to conduct all of its activities in accordance with applicable antitrust and competition laws. It is, therefore, extremely important that attendees adhere to meeting agendas and be aware of and not participate in any activities that are prohibited under applicable US state, federal or foreign antitrust and competition laws.

If you have questions about these matters, please contact your company counsel or counsel to the Joint Development Foundation, DLA Piper.

Recordings

WG agreed to record all Meetings. This meeting recording will be available until the next scheduled meeting. Meeting recording link

Roll Call

Please add 'Attended' to this issue during the meeting to denote attendance.

Any untracked attendees will be added by the GSF team below:

Agenda

Introductions

PR Reviews

Project Dashboard

AOB

Future meeting Agenda submissions

Next Meeting

Adjourn

Meeting Action Items / Standing Agenda / Future Agenda submissions

seanmcilroy29 commented 7 months ago

MoM

Software carbon efficiency standard. Chris presents a software project for measuring carbon efficiency and proposes a certification framework that includes categorization, benchmarking, and labelling of software companies.

Sustainability in large language models. Chris and Boris discussed the potential of Large Language Models (LLMs) in AI, specifically in energy consumption and sustainability. Chris is working on a plan to understand the situation and develop a certification process for LLMs. During the conversation, Chris expressed disappointment over the lack of prominent sustainability metrics, such as energy consumption, provided by the Hugging Face model repository for their models. Boris agreed that standardisation is a major gap in the field and suggested an opportunity for collaboration to address this issue.

Developing a standard for measuring AI sustainability. The development of AI sustainability strategies is a growing concern, mainly due to a lack of standardisation in the industry. Chris and other experts are discussing the creation of a base class for LLMs, focusing on standardised benchmarking and efficiency. Tammy has expressed confusion about the project status, but Sean has provided an update on rating system test cases with Ella Lambs.

AI rating system for the ecosystem industry. Chris provided an update about the new standard for the ecosystem industry. The focus is on iterative software methodology and collaboration with a working group. During a discussion with Tammy about the rating system for Ella LEMs, Tammy expressed concerns regarding the feedback. Sean suggested that they finalise the baseline for case studies soon. Sean plans to share this information with the group and encourage more people to review the specifications and provide feedback.

Standardizing energy efficiency units for AI models. Boris discussed two platforms that can be used for evaluating the energy efficiency of language models - Hugging Face's LLM perf leader board and ml energy from the University of Michigan. During the discussion, Boris highlighted the challenges in standardising energy consumption reporting for AI models due to variations in units and hardware. Collective decision-making in open-source development. Chris proposes involving all stakeholders in collective decision-making to address gaps and challenges in the Spec.

Standardising carbon footprint measurement for AI models. Tammy and Chris discussed the SCI's adaptability and how it can be applied in different cases. They discussed defining software functions and measurement units for large language models and using kilowatt hours or CO2 equivalent in sustainability benchmarking. Later, Tammy and Sean discussed the need for a thorough discussion on the new V 1.1 standard for the SCI, with Chris and Tammy contributing their views. Chris and Tammy also emphasized the importance of involving more people in the conversation to make the standardisation process more comprehensive and iterative.

Salesforce membership and green AI. Sean seeks follow-up on Salesforce membership and the GreenAI aspect, with potential for collaboration. Open-sourcing software for renewable energy standards. Chris and Boris agree to open-source their models and collaborate on SCER software development.

AI standardisation and collaboration. Sean and Chris discussed publishing a draft standard for the GreenAI initiative. The objective is to submit the draft to the SWG by the end of May for community feedback. The group discussed responsible AI and its definition. Chris has contributed to the LF AI data WG and helped promote their work.

Action Items