search

w3c / ai-accessibility

https://w3c.github.io/ai-accessibility/
https://w3c.github.io/ai-accessibility/
Other
10 stars 3 forks source link

AI and Machine Learning - note fails to discuss sustainability aspects of using AI #15

Closed aardrian closed 1 month ago

aardrian commented 2 months ago

Issue #13 touches on this, but I think it warrants its own section in addition to ethical aspects. While the Web Sustainability Guidelines are structured as a recommendation and testing framework, I think it can offer some language or principles to incorporate directly.

In particular, some acknowledgment of the impact on end users when this work is offloaded to their browsers / devices. Particularly as alluded in the second half of Section 5.

Here is a very brief list of references, though they deal mostly with the current crop of generative systems and not so much with the generally more sustainable machine learning technologies that pre-date genAI:

I don't mean to suggest these all would be added to Section 6, but right now that section has no references addressing sustainability.

frankelavsky commented 2 months ago

Holy smokes, this is thorough. And I actually appreciate that these focus on GenAI predominantly, because this problem is new and pernicious and specific to the current state of things. Of course, we shouldn't be focusing on only GenAI, but we want to make sure that however we formulate our language and framing, we remain critical of the present and future sustainability of the technologies we employ.

Mizokuiam commented 2 months ago

Totally agree that sustainability needs its own section. It's definitely important to address the potential impact on end-users, especially with more processing being offloaded to client devices. Section 6 on ethical considerations seems like a natural place to add this, maybe as a subsection, since there's obviously overlap.

I've seen similar issues when discussing accessibility best practices – sometimes performance gets overlooked. We want to avoid creating guidelines that inadvertently encourage unsustainable practices. Referring to the Web Sustainability Guidelines makes a lot of sense. It's a good starting point for consistent language and principles. Plus, it helps align our recommendations with broader web standards, which is always a good thing.

Regarding your references, wow, that's quite a list! While it focuses heavily on GenAI, as @frankelavsky mentioned, that’s not a bad thing given its current impact. We can certainly pull key points and principles from those articles, but we should probably be careful not to over-index on GenAI specifically. The core concepts around energy consumption, water usage, and e-waste apply more broadly to ML models, even pre-GenAI.

Maybe starting with a paragraph outlining general sustainability concerns related to AI/ML, then having a separate paragraph focusing on the specific implications of GenAI (with a subset of your references), might be a good approach?

Let me know what you think.

Cheers!

aardrian commented 2 months ago

@Mizokuiam

Maybe starting with a paragraph outlining general sustainability concerns related to AI/ML, then having a separate paragraph focusing on the specific implications of GenAI (with a subset of your references), might be a good approach?

I think it goes beyond the sustainability. That is core, of course. But the increased energy demands are sometimes offloaded to end users directly. I touch on this in #16 and how overlays are a good example of this. When end users are the ones remediating content using genAI (or algorithms or whatever), that means each end user who performs that fix absorbs the burden -- whether through bandwidth to get the overlay's code, battery / electric usage for their browser to perform the remediation, and then the repeat costs when it happens on every new page and every visit.

Multiply that by every visit to the page generating its own remediation from an overlay, and the site has essentially initiated a distributed over-use of electricity attack on its users and, well, everyone.

To recap, it's about covering the known demands of genAI and also how the casual use of genAI when not necessary acts as a multiplier.

mgifford commented 1 month ago

There are times when AI is the right tool for the job. Then there are times that isn't. Finally, there are times that the job shouldn't be done at all. Ever.

In the case of alt text, it is clear that AI is going to be incorporated into more and more tools, and that it will be used multiple times if good alt text isn't provided.

There are things that humans just aren't good for, like writing good alt text.

I'd much rather have AI used once by authors to provide a quality result (with the review of the author) than even a hundred times by different AI tools.

aardrian commented 1 month ago

There are things that humans just aren't good for, like writing good alt text.

As a blanket statement, I disagree with that. Humans can be very good at it, even with no or minimal training.

I'd much rather have AI used once by authors to provide a quality result (with the review of the author) than even a hundred times by different AI tools.

A single initial pass, I agree, is far less problematic than generating content on each page load (which is the modal overlays promote). However, those aren't the only two options and pre-emptively excising humans from the process — especially those with training and / or context — should not be the default starting position from a sustainability perspective.

If we approach sustainability concerns using a false dichotomy fallacy then we won't make much progress on guidance.

joshueoconnor commented 1 month ago

Discussed and the consensus was this is out of scope with RQTF - we defer to the W3C ethics (and other) docuṃents for any discussions on sustainability.

aardrian commented 4 weeks ago

@joshueoconnor

  1. Are the minutes of the discussion public (and if so, can you share a URL)?
  2. What is "RQTF"?
joshueoconnor commented 2 weeks ago

@aardrian Minutes are member only I'm afraid, and RQTF is the 'Research Questions Task Force' where this work is taking place.

Thanks

stevefaulkner commented 2 weeks ago

@joshueoconnor Why are the minutes member only?

stevefaulkner commented 2 weeks ago

@aardrian @joshueoconnor

looks like the RQTF minutes are public https://www.w3.org/WAI/about/groups/task-forces/research-questions/minutes/

joshueoconnor commented 2 weeks ago

Sorry @aardrian I meant you may need a W3C account. Thanks @stevefaulkner for finding the URI for the public list.