Reference Architecture and Best Practices for Sustainable AI with Cloud Native Environments

[zanetworker]

We would like to produce a reference architecture and best practices for sustainable AI with cloud native environments. This will factor in the AI/ML lifecycle.

• The document will should aim to discuss the needs for sustainable AI practices.
• The document will emphasize the challenges posed by AI's increasing computational demands.
• The document will also look at the "good guy" side of cloud-native and their positive impacts on AI scalability, flexibility, resource efficiency, and resilience.
• It will outline the core components of a reference architecture for sustainable AI, covering compute resource management, storage optimization, networking and connectivity, and sustainability metrics and monitoring.
• It will provide best practices and recommendation, along with insights into future trends and research directions.
• The document will finally provide key insights and will advocate for the integration of sustainable best practices for cloud native AI deployments.

cc @rootfs @terrytangyuan @caldeirav

Starting document: https://docs.google.com/document/d/154mBlTykdZPrR11TmvMjffv0-JZHDT82zN2blmg15Qk/edit

Probably related to:

• https://github.com/cncf/tag-env-sustainability/issues/372#issuecomment-2124940573
• https://github.com/Green-Software-Foundation/gaic/issues/10

[zanetworker]

Cross linking: https://github.com/cncf/tag-env-sustainability/issues/372#issuecomment-2124940573

[caldeirav]

Highlighting the importance of this topic, the EU Artificial Intelligence Act outlines a comprehensive approach to ensuring that AI systems developed or deployed in the EU adhere to environmental sustainability standards.

Here's a breakdown of these key articles for reference:

Article 4a(f): This article introduces the principle of developing and using AI systems in a sustainable and environmentally friendly manner. It underscores the need for AI technologies to be designed with environmental considerations at the forefront, promoting a green technology initiative across the EU.

Article 12(2a): This provision specifies that high-risk AI systems must have capabilities for logging energy consumption and for measuring or calculating resource use. Providers of these systems are also required to assess the environmental impact throughout the lifecycle of the system. This includes not just the operational phase but also the design, development, and decommissioning stages, ensuring a comprehensive overview of the environmental footprint of AI systems.

Article 28b(2)(d): For providers of foundational models, this article mandates adherence to standards that focus on reducing energy and resource use, improving energy efficiency, and enabling the measurement and logging of environmental impacts. The provision extends the responsibility of environmental stewardship across the entire lifecycle of the AI model, from inception through deployment, which significantly broadens the scope of environmental accountability in AI development.

Article 29a(g): This article requires that fundamental rights impact assessments for high-risk AI systems include measurements of reasonably foreseeable adverse impacts on the environment when the system is put into use. This addition ensures that environmental considerations are integrated into the broader framework of rights and impacts assessments, aligning AI system deployment with the EU’s environmental protection goals.

These articles collectively contribute to a regulatory framework that not only promotes environmentally sustainable practices in AI but also aligns these practices with the broader goals of fundamental rights protection and sustainable development within the EU. However the requirements are broad in nature and helping organisations to standardise their implementation approach on open source technologies including models now will also have regulatory importance.

[catblade]

Copied the doc over to the env sust drive: https://docs.google.com/document/d/1tNfR5yxnqfu9fduEb3108oBGXQ6spyRzA6cBTs9zbnQ/ and will use this as a working copy.

[zanetworker]

Had our first meeting today, notes can be found here: https://docs.google.com/document/d/1y0aCen7b0guvVAnqgPCb_lil71YvjhyZiXzyn1iC8qs/edit

[zanetworker]

Paper is now being edited here: https://docs.google.com/document/d/1tNfR5yxnqfu9fduEb3108oBGXQ6spyRzA6cBTs9zbnQ/edit