Open glipstein opened 2 years ago
Chatted with Peter, reflecting a couple thoughts to consider:
Relevant paper on speed of algorithmic decision-making; "Decision Time: Normative Dimensions of Algorithmic Speed" https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4078006
Context from article where this was linked:
As Daniel Susser argues in his recent paper, the speed at which automated decisions are reached has normative implications. By incorporating digital technologies in decision-making processes, temporal norms and values that govern them are impacted, disrupting prior norms, re-calibrating balanced trade-offs, or displacing automation’s costs. As Susser suggests, speed is not necessarily bad; however, “using computational tools to speed up (or slow down) certain decisions is not a ‘neutral’ adjustment without further explanations.”
Another place this is coming up is LLMs (this came to mind as a potential gap from the part of the AFD post on LLMs + deon). One of the clear issues with deployment is the ease and scale of generating content for things like school essays or peer review forums (e.g., as mentioned here) which are not as prevalent as one-off issues of misuse.
That said, I'm not totally sure what the approach would be to address this. cc @jayqi @ejm714 @pjbull if any thoughts while we're on the topic. Especially thinking about deon with generative AI as opposed to traditional predictive algorithms
Adding from @pjbull note on feedback loops in generative AI - chain of thought, risk of signal degradation, e.g., chaos GPT?
It may be that this item focuses more on feedback loops (including reinforced outcomes per first example above), and less on speed/scale
Following up from comment thread in PR #140
A few thoughts below, if we think this is worth adding. E1 is related but this feels like a sufficiently different question to consider, especially with everything that's already present in E1.
Possible wording:
E.5 Systematic effects (could also be, say E.3 and push the other two to 4/5)
Have we considered risks posed by the scale, speed, or rigidity of the deployed model that aren't present in the equivalent human or prototype process (e.g., reinforced outcomes and feedback loops, ability to consider missing variables, societal impacts)?
Possible examples:
I think this example of feedback loop (formerly of "concept drift") actually fits in here; this is a result of using the model, not just a distribution that happens to shift on its own
curious if folks have thoughts on other examples to represent broader systemic impacts that might not be captured elsewhere in the checklist. possible example: speed of misinformation spread + twitter (Science article, Twitter's crisis misinformation policy to slow down viral tweets, etc.)