Create DyME case study to link with Bias Card Activity framework (TEA?)

[dingaaling]

• HEV framework
• https://alan-turing-institute.github.io/turing-commons/skills-tracks/rri/rri-203-4/#cognitive-bias
• Case study: https://github.com/alan-turing-institute/turing-commons/blob/resources/resources/case-studies/air-quality.pdf
• Case study DyME: https://hackmd.io/wTGdj2_3R1eE6lUJqWIy-A

Proposed process:

• [x] complete small piece of desk research, collecting challenges & solutions for fair automated decision making
• [ ] map those likely abstract findings onto HEV framework specifically
• [x] consult with @chrisdburr about identified ethical principles & which ones are best suited for assurance

[dingaaling]

Notes from 1 Dec convo:

• It’s easy to take action based on data; harder to understand if it’s the right data
• Automated decision making optimised for prediction
  • Integrating the states/actions you can take into the assessment of risk
• Assurance doesn’t give the answers, but it gives a process to formalise the process

[aranas]

• Automated decision making optimised for prediction
  • Integrating the states/actions you can take into the assessment of risk

Great Summary! Some more quick thoughts (to be worked out more in detail):

This is the paper I mentioned, that describes the extensive critique on automated decision making systems that are optimized for prediction Wang, Angelina, et al. "Against predictive optimization: On the legitimacy of decision-making algorithms that optimize predictive accuracy." Available at SSRN (2022).

some points made in the paper relevant for the HEV framework:

• Interventions might inadvertently perpetuate existing inequalities (read vulnerabilities), especially when they rely on biased data or fail to accurately represent the target population.
• Decisions based on predictions can affect the outcomes being predicted, potentially creating self-fulfilling prophecies that further harm vulnerable groups. (explored in the paper mainly with focus on crime, not sure if will translate to climate risk)
• The aggregate of individually optimal predictions may not lead to globally optimal interventions, overlooking broader social impacts.
• Dangers of "distribution shift" relevant if framework is to applied in multiple countries

  "The most common [distribution shift ]is that the training population is different from the target population in important ways, e.g., due to differences in geography."

• Dangers of "target-construct mismatch", is occurrence of weather hazards the target? Or are we relying on proxies to measure outcomes?

  "where the variable used as a proxy for the desired outcome does not accurately represent it, can lead to inaccurate and biased predictions."

Some fairness and ethics considerations mentioned in the paper:

• Achieving fairness in predictive optimization is complex and involves political and ethical considerations beyond technical solutions.
• Different moral dimensions of fairness are captured by various statistical fairness criteria (eg factual exposure stats), but these do not address deeper societal inequalities.
• Goodhart’s Law: Strategic behavior by individuals can undermine the accuracy and validity of predicted vulnerabilities/exposures

practices towards ensuring solutions target vulnerable demographics that could be "assured"

• Contestability
• Transparency

[aranas]

And here the other paper that continues this line of research but then argues for taking into account the action space: Liu, Lydia T., et al. "On the Actionability of Outcome Prediction." arXiv preprint arXiv:2309.04470 (2023).

haven't read yet but might be interesting in that context. From the abstract: "Making measurements of actionable latent states, where specific actions lead to desired outcomes, considerably enhances the action value compared to outcome prediction, and the degree of improvement depends on action costs and the outcome model. This analysis emphasizes the need to go beyond generic outcome prediction in interventional settings by incorporating knowledge of plausible actions and latent states. "

[aranas]

Have started to draft a case study to be completed with Ruth & Alisha here: https://hackmd.io/wTGdj2_3R1eE6lUJqWIy-A

[chrisdburr]

Very helpful to have these notes shared and some of the links brought together.

Seeing the HEV framework, especially, is particularly instructive as it provides an additional framework onto which specific biases (and ethical issues more generally) can be mapped.

For instance, representation bias could obviously occur between the hazard identification and exposure stages. Similar to an example I used in a talk at the Met Office last year.

Figure 1—HEV framework

Figure 2—graphic showing how placement of radar sensor to detect storm risks, unintentionally created racial bias by underrepresenting black communities