04/28: Sandra Wachter

[ehuppert]

Comment below with a well-developed question or comment about the reading for this week's workshop. These are individual questions and comments.

Please post your question by Wednesday 11:59 PM, and upvote at least three of your peers' comments on Thursday prior to the workshop. You need to use 'thumbs-up' for your reactions to count towards 'top comments,' but you can use other emojis on top of the thumbs up.

[sabinahartnett]

Thank you for sharing your work with us Professor Wachter!! I really enjoyed the reading on AI in EU non-discrimination law . Something that became apparent to me throughout this reading was the tension between public interest (assuming that aligns with non-discrimination) and private interests of the companies often charged with (or who undertake because of the market potential of a product) creating these AI - in advertising the models they would likely be most interested in proving efficacy (this may match the statistical evaluations you suggested in your text) of their model in order to sell it. My question is thus: what does the current competitive landscape look like in Europe for AI implementation in discrimination law? And what would be your suggestion for ensuring the most 'neutral' creation of these tools?

[pranathiiyer]

Thank you so much for sharing your work Professor! I wanted to ask you about something that you touched in your paper on bias preservation in machine learning which I thought was very important. You talked about how dependence of ML systems on ground truth labels and counterfactuals for evaluation makes it difficult to see if a shift occurs between the training and deployment of the model. While there are ways of using certain proxies and active learning to counter these, I would think these can themselves induce systemic bias. How do we tackle problems of this kind which make it difficult to even judge the performance of our model?

[JadeBenson]

Thank you so much for sharing your work! It was interested to read about the differences between discrimination law in the EU and the US and how this has ramifications for fair AI practices too. At the end of your paper you argue that "we can only overcome this challenge together." I agree - I think we need expertise from a variety of fields in order to understand how to address this complex of an issue. However, I'm wondering if you have any recommendations for creating these coalitions? It seems that data/computer scientists can be siloed in their own teams and legal only gets involved if a claim escalates to a certain point. How can we better incorporate teamwork between technical and legal groups into businesses and academic research?

[Thiyaghessan]

Hi Professor,

Thank you so much for sharing your work with us. The case for statistical "gold" standards has been convincingly argued for in your paper. However, I cannot help but fear that the implementation of such legislative benchmarks can quickly become politicised by bad-faith actors. Reading the description of conditional demographic disparity, I can already imagine the partisan attacks that will be conjured up in response, deploring how majority groups are being unfairly persecuted. This phenomenon is of course nothing new but in the context of algorithms, it can allow tech companies to avoid scrutiny by taking advantage of the politically polarised environment, as is the case in the United States now. This problem becomes more severe if we include the potential for simpsons' paradox type situations where biases are revealed to be less prevalent than once conceived of. How do we get the public to develop the statistical understanding necessary for them to support these laws?

[afchao]

Thank you for sharing this work with our group! The topic of addressing algorithmic biases inherited from systemic social biases grows increasing important as we offload more of the work of running our societies to the robots, but is there a sense in which these efforts are more along the lines of replacing one bias with another rather than eliminating bias from e.g. jurisprudence altogether? I naively believe that a truly unbiased system of law would be ideal, but unpacking that belief reveals some measure of subjectivity (e.g. I would consider a system which disproportionately favored historically disadvantaged groups as less biased than a literally egalitarian one if implemented tomorrow), and while I could advocate and maybe even make a decent argument for why my interpretation of e.g. the law's purpose is superior, aren't we still just shuffling biases around the table at the end of the day?

[chrismaurice0]

Thank you for coming to our workshop! Your piece focuses on the incompatibility between European notions of discrimination and existing work on algorithmic/ automated fairness in addition to the problems with EU's current law, which your paper state " does not provide a static or homogenous framework suited to testing for discrimination in AI systems." I remember when Mark Zuckerberg testified before the U.S. Senate, which revealed Senators' lack of social media knowledge-- putting into question their ability to create effective regulation. Do you think the reason for a gap between EU non-discrimination law and AI exists because the lawmakers writing such laws are unfamiliar with the nuances of AI?

[ValAlvernUChic]

Hello Professor!

Thank you so much for sharing this work with us. What particularly stuck out to me was your argument on intersectional discrimination and ways that the concept could be used (i.e. negative dominance and divide and conquer) to work against the very communities the concept was made to protect/consider. Having worked in a legal organisation that advocates for foreign domestic workers, I saw often how the multiple dimensions to these workers' identities were weaponized against them. Beyond their ethnicity and gender, they were additionally judged by their status as transient workers. While the papre focuses on the EU and its members states/workers, I was wondering how we can think about including these communities who often are not afforded the same legal protections in a country, despite being subject to the same expectations and systems?

[LynetteDang]

Hi Prof Watcher,

Thank you so much for sharing your work with us. I am curious to see your stand on the possibility of having software tools and development platforms that will help engineers ensure their AI products are free of bias, and ensure fair, explainable, transparent and accountable AI systems? And how long do you think the process it will this take? Will we be able to see this in our lifetime? I have been following bias-elimination algorithm such as IBM's AI Fairness 360, do you think these algorithm truly allow us to take bias out of the picture?

[Yutong0828]

Thank you very much for sharing your work with us! It was mentioned that most of the previous studies have focused on American non-discrimination laws, so what are the main differences in non-discrimination laws in the EU compared to America? For example, are there more laws in EU that address discrimination between countries? Also, I was wondering if there are algorithms that can be used to predict the effect of adopting a particular anti-discrimination practice? For example, in addition to assessing whether organization A discriminates against group B, can algorithms (e.g., simulation algorithms) be used to help determine exactly what methods should A use to reduce discrimination and their effect? In particular, when we consider historical inequalities for “bias transforming” metrics , how can we estimate whether these historical inequalities can be reasonably addressed by improving specific policy practices?

[atowey-uchi]

Thank you for sharing your work with us! I was particularly interested in the delineation of additive discrimination and intersectional discrimination and how these concepts rely on socially constructed identities. I am curious to see how this delineation is handled in automated technology versus how it is handled in a legal setting. Judges are humans and have their own inherent biases and automated technology can, in some ways, ameliorate such differences in biases between judges. Automated technology, however, is programmed by humans and takes in human data, which itself can induce biases and discrimination. How could a "meeting of the minds" between tech and legal help to challenge the inherent biases in each system?

[bowen-w-zheng]

Hi Prof Watcher,

Thanks for sharing your work! So glad to see the notion of algorthimic fairness is getting some attention it deserves. I am interested in your take on how should we go about deciding what notions of fairness are most appropriate? Specifically, what do you think would be a legitimate and effective process of choosing the right notion? For instance, which branch of the governing body should be granted the power of choosing and enforcing such policies and on what ground? How do we engage more people in the formation of such notions and decisions?

[borlasekn]

Thank you for sharing your work with us! I was wondering what you believe best practices are to ensure that this interest in algorithmic fairness in AI extends beyond academia? For example, how do you ensure transparency for all entities using AI, especially while upholding rights and freedoms that these entities may say that they are owed?

[Yaweili19]

Western societies are marked by diverse and extensive biases and inequality that are unavoidably embedded in the data used to train machine learning. Algorithms trained on biased data will, without intervention, produce biased outcomes and increase the inequality experienced by historically disadvantaged groups. Recognising this problem, much work has emerged in recent years to test for bias in machine learning and AI systems using various bias metrics. In this paper we assessed the compatibility of technical fairness metrics and tests used in machine learning against the aims and purpose of EU non-discrimination law. We provide concrete recommendations including a user-friendly checklist for choosing the most appropriate fairness metric for uses of machine learning under EU non-discrimination law.

Thank you for sharing this piece of amazing work. Your review on the technical fairness metrics and tests is truly beneficial for researchers working on ethically sensitive topics, myself included. I also especially enjoy your concrete recommendations and the checklist. They are making my work so easy.

My question here is, how reliable are these assessments if we try to generalize it to other methods/algorithms, like NLP methods and deep learning?

[PAHADRIANUS]

Thank you for sharing your work and pointing out the potential legal implications of machine learning's discrimination issues. 1) As you admitted yourself, high accuracy modeling often seeks to replicate with precision the decision making in the training data and thus preserve their biases, and thus may favor bias preserving than bias transforming metrics. There is a contest between formal and substantive equalities. I wonder in specifics how the trade-off should be made: would decision makers be willing to invest in the more costly method of reducing discrimination or rather they would simply stick to the formalities to satisfy legal requirements? 2) Continuing from the assumption that decision makers might be more inclined to only satisfy formalities, how do you think anti-discrimination legislation should be modified in order to cope with the situation to enforce more substantial equality?

[jinfei1125]

Hi Prof. Wachter, thanks for coming to our workshop! As discussed in the paper, "social biases" are very difficult to fix, and there are many questions about "substantive equality" such as what is the end goal of non-discrimination law, to rectify historical harms, combat traditional power hierarchies or achieve equality of distribution of goods for all. Since fairness cannot be automated and the status quo algorithmic decision-making can only not be neutral. I also think it's very helpful to provide the user-friendly checklist for choosing the most appropriate fairness metric for uses of machine learning and AI under the EU non-discrimination law. Do you anticipate any practical use of the checklist for lawmakers in the US or EU?

[taizeyu]

Dear Prof. Wachter, thank you very much for sharing your research, My question is if the standard of non-discrimination similar all over the world? In other words, if the result of the research can be applied to another country, like the US.

[Raychanan]

Hi Prof. Wachter, to me the most argument the paper made was that algorithmic systems lack equivalent mechanisms and agencies for signaling. The concept of automated discrimination differs from traditional forms of discrimination in that it is more abstract and intangible, subtle and difficult to detect. It is evident, however, that deep learning scholars are diligently studying, visualizing, and eliminating bias from deep models. Therefore, the argument was surprising at first glance. I am interested in your views regarding debiasing work and their possible application to litigation. Thanks!

[helyap]

Dear Prof. Wachter,

Thank you for sharing your work with us! It appears that much of the legal foundation regarding AI/ML fairness and ethics is centered around enforcing policies from the technical-producer side or higher-level legal entities (i.e. technologists, software companies, and then the legal system and courts). The recommended standards also appear largely dependent on judiciary branches. Considering the heterogeneity across Europe, for example, in identity and equality policies, sentiment, and norms, do you foresee challenges in the actual implementation of the CDD measurement? For example, according to Brookings*, there is a 1978 law in France that "specifically banned the collection and computerized storage of race-based data without the express consent of the interviewees or a waiver by a state committee" and race appears to be a taboo topic to discuss. Will such precedent then challenge the ability to implement the usage of CDD? Also, will proper enforcement and implementation of the standards be contingent upon impartial adjudicators and evaluators? Finally, with considerations of the “black box” problem, do you foresee any possibility in policies and practices that can empower the users of AI/ML (and not just the technical/legal communities) in assessing fairness and non-discrimination and to reclaim intuition?

*https://www.brookings.edu/articles/race-policy-in-france/#:~:text=Race%20is%20such%20a%20taboo,or%20ethnicity)%20of%20its%20citizens.

[hhx2207061197]

Hi, Professor Wachter. My question is, do you think the standards of non-discrimination are similar around the world? How can we generalize this concept? Is there any heterogeneity among different countries and regions?

Thanks a lot!

[MengChenC]

Hi Professor Wachter, thank you for sharing your work, and there has been surging research and debate on how should machine learning and AI facilitate juridical cases. Specifically, researchers and the field argue much regarding the automated percentage for AI in the justice domain. For the algorithms trying to mitigate the bias in every law case, how many proportions can we rely on AI, how should we justify the allocation, and ensure the introduction of AI will not backfire on the justice system? Thank you.

[wanxii]

Thank you so much for sharing the work! I wonder how we should handle the tradeoffs between the efforts to guard nondiscrimination in algorithms and the potential loss of perdiction accuracy / suboptimal selection outcomes. Thank you!

[egemenpamukcu]

Hi Professor Wachter, thank you for sharing your work. I am interested in whether it would be possible to integrate legal processes that could scale with the number of potential cases that could be generated through social media. Regardless of where we draw the line in terms of bias we want to prevent and harm we want to limit, it seems to be like there will always be political backlash that will undermine the fairness principle of any "algorithmic justice" framework. Similar, but definitely not the same, disputes have historically been solved through litigation and arbitration. It is obvious that with existing institutions and infrastructure it would not be possible to relax concerns of bias in our algorithms and adopt a case-by-case approach where each case would be handled using existing legal processes. But, maybe we can update the legal system using tech to handle disputes and wrongdoings in the digital spehere at scale. Sounds like a long shot but I was wondering what you would think about such a future.

[ChongyuFang]

Hi Prof. Wachter,

Thanks for sharing us with your work which introduces to us certain legal implications of the discrimination issues we faced in the field of machine learning and artificial intelligence. My question is: for the CDD method you proposed, do you have any thoughts in mind to address the issue of biased training set (which in turn gives biased outcomes), and improve it such that it can be used for different sectors? Thanks!

[NikkiTing]

Thank you for sharing your work! Your paper on bias preservation in machine learning was really an interesting read. How do you think the results of machine learning and AI models that satisfy bias transforming fairness metrics compare with results from human assessments in terms of creating fair outcomes? Given the social reality of systemic bias and inequality, human decision-makers are also prone to making biased decisions and retain the status quo even with non-discrimination policies in place. Do you think fair machine learning can lead to less biased decision-making outcomes than humans, perhaps in the future?

[XinSu6]

Thank you so much for sharing your work! As you mentioned, the social biases are hard to be identified and to be eliminated in your paper's context. I am wondering how true is it in other fields? Have you tested your tools and methods anywhere else? And generally speaking, how long do you think it will take for AI products and platforms to be considered fair and bias-free universally? What are some of the possible trade-off?

Thank you and looking forward to your presentation.

[BaotongZh]

Dear Professor Watcher Thank you for sharing us such great works, the topics in the paper really gives me a brand new perspective to view the gap between the legal systems and automated metrics of algorithms. My question is while the efforts is crucial to make the legal system more robust to deal with the algorithm discrimination, what are your opinions on how can we make normal people aware of the existence of such discrimination or bias and what kind of metrics can we apply.

[yjhuang99]

Hi Professor Wachter, your work is truly amazing with the aim of reducing bias in ML and AI algorithms and promoting fairness, especially from a legal perspective. My question is while reducing technical bias is the main goal of most fair ML research, would you provide an example that the CDD schedule can be applied to correctly measure the true bias in the setting of this paper? Furthermore, can this metric be widely adopted in any field? Thanks!

[ginxzheng]

Thank you for coming Professor Wachter. I am curious about your discussion of the EU non-discrimination law. May I wonder would you have any empirical evidence of how and why automated discrimination is harder to detect? Why it's unintuitive and abstract? Why do you think the non-discrimination law should be designed in a way homogeneously suited for AI instead of leaving the heterogeneity case by case? Thank you!

[xin2006]

Thanks for sharing your paper! I found the viewpoint really valuable that AI is profoundly changing the format, spread and impact of discrimination, which is covert but efficient, so it's necessary to resort to legal protection. For the automated discrimination and legal protection in your paper, I have two questions. Firstly, I'm curious whether the data source could be one reason of discrimination? For example, there is less data for the elder than the younger. Is it possible that the bias due to lack of data would cause discrimination? Secondly, I'm a little confused about what is the strength of consistent procedures for assessment over judicial interpretation? Are we doing something like using algorithm to assess algorithm?

[zhiyun0707]

Hi Professo Wachter, thanks for presenting your findings! In the paper Bias Presentation in Machine Learning, you mentioned on how the status quo is not neutral and many of the limitations come from the neutral starting point to assess fairness in machine learning. You gave people's making decision of inequality and prejudice in Western society on gender, race, and social stigma around disability for explanation, and how machine learning may have bias correspondingly. I wonder that if we change the place to Eastern societies that may have different sets of bias, how will the study results change? Thank you!

[ShiyangLai]

Hi Prof. Wachter, Thank you for sharing. In my point of view, people's definitions of discrimination are extremely unstable in different contexts. For example, if we are doing customized products recommendation, for me, it makes sense to preserve some kinds of "discrimination" within the recommendation model with the aim of distinguishing different people's needs. I am wondering, in these types of scenarios, can we really find a universally applicable non-discrimination law? And, in the long run, what is the meaning of creating non-discrimination laws? Is our definition of discrimination really something that needs to be regulated/unified?

[YLHan97]

Hi Professor Wachter,

Thanks for sharing your research with us. I have a question as follows: In the article “Bias Preservation in Machine Learning: The Legality of Fairness Metrics Under EU Non-Discrimination Law”, you have mentioned that the legal need for justification in cases of indirect-discrimination can impose additional obligations on developers, deployers, and users. I’m really interested in the machine learning applied in real world, but not so much familiar with the law industry. So, would you please provide more real world examples in your relevant area?

[yujing-syj]

Hi Professor Wachter, thanks for sharing your research with us! I am very glad to hear this interesting topic that combines the social science issues and AI technology. My question is, since you mentioned that automated discrimination is abstract, unintuitive and intangible, whether there will be potential problem that it is hard for many communities to work together to have a common idea? What are the existing problems that hinder the collaboration among the intelligent groups?

[JunoWuu]

Hi Professor Wachter, thank you for sharing your work with us. I think it makes total sense that our intuitive thinking cannot be enough for making legal judges. And you mentioned that as the problems get more complex, the more we will find our intuitive thinking would not be enough. Do you know where the line is? when can we be sure that automatic factors will be needed?

[javad-e]

Thank you for presenting your work at our workshop! I was wondering what your thoughts are on distinguishing between discriminating algorithms and algorithms that reflect real-world inequalities. As the examples in Wachter et al. (2021) illustrate, we must pay attention to potential biases. Discrimination against the African-American population by the police is a great example. Of course, we must be careful correcting the machine bias, however, if due to prior economic and historical events, the crime rate is higher in some African-American communities, we do not really want the machine predictions to ignore this higher rate. So, in such cases where equality is not the goal, how can we decide if the algorithm is truly biased?

[zixu12]

Hi Professor Wachter, thanks for sharing your inspiring work, which I believe a lot of people care. My question might be a bit detour of your research: what is your view on algorithm aversion, which means average people tend to not trust what the algorithm provides compared to their own experience. Do you think it can be a back force for the fairness of the ML? Thank you!

[Emily-fyeh]

Hi Professor Wachter, thank you so much for sharing your latest work with us! I really like the part you discussed the dilemma between EU-centric humanity values and commercial benefits in the tech era. I would want to know other than the nominal ethical principles rooted in the legal practice and legislation (which should be expanded to the modern AI setting), what are other possible ways to provide incentives to the private sector to devote more to the non-discrimination practice? (Especially starting from the inner standards of the corporations)

[hshi420]

I was wondering what would be the most practical ways to improve the non-discrimination AI. Also, since there can be different discrimiation types all around the world, how could you generalize your improvement methods?

[NaiyuJ]

Hi Professor Wachter, This is an amazing project! Very promising and down to the reality. I'm more interested in this topic of protecting fairness in the AI world in a comparative sense. That is, what would be the differences between the U.S. context and EU context, not for now, but in the future. Will all countries tend to converge to the same set of actions?

[JoeHelbing]

The conscious and unconscious biases of humans are (mostly) based on some understandable and individually identifiable characteristic such as race, gender etc. There are many protections enshrined in law such you mentioned that specifies these groups and provides legal protections. In your papers you mentioned that AI and ML may create biases and discrimination against new groups, and that these groups might not fall under our current understanding of protected groups.

It seems that the fine grained ability of ML to tease out patterns from data and make decisions on access to any number of things, jobs, loans, grants, scholarships, school access, that it has the potential to create not a set of defined groups, but a spectrum of different characteristics that ML chooses against. Laws work on a binary nature, this is legal, that is not, but given the granularity of AI, how do we draw a line between an AI making efficient choices, and an AI making discriminatory choices?

There are bright line issues that society mostly agrees on, but AI complicates that immensely with the depth of characteristics it works against and the fineness by which it applies weights to those. I wonder how society can come to agreement that say a negative discrimination of 0.24x^2 against Body Mass Indexes above 32.2% is or is not discriminatory.

[boyafu]

Hi Professor Wachter, thank you so much for sharing your work! I enjoy the discussion about the distinction between equal opportunities and the recognition of the aim of laws. Given the ever-changing nature of algorithms, models, and the derivative ideas of equality, I wonder what role would legislation play - building fairness metrics with predictions or reacting to the emerging inequality patterns. Thanks!

[xxicheng]

Hi Professor Wacher. Thank you for sharing the fascinating work. My understanding of biases is always group-specific. In other words, depending on the data collection, there might be various biases and discriminations existing in the algorithm. How should we deal with this heterogeneity?

Also, the opening sentence of the abstract interested me. "Western societies are marked by diverse and extensive biases and inequality that are..." I am VERY curious about your perspectives on what specific kinds of "diverse and extensive biases" make them one of the characteristics of Western societies. Does the word "marked" imply a comparison between the West and the East? What do you think about the West/East dualism? I might sound very speak-like-a-book, but I think as we are talking about eliminating bias, it is not wrong to be sensitive about Euro-Americianism. To be honest, I think this sentence is really more appropriate as a conclusion rather than as an opening. Looking forward to your response!

[YaoYao121]

Hi Professor Wachter, thanks for sharing your work with us! That's really an interesting question at the interaction of the hot social science issues and the promising technology. My question is focus more on the ethical aspect. I am a little curious about, since human beings ourselves have such a long history with discrimination and still tend to make decisions with discrimination in social activities, if there is a perfect AI algorithm can generate non-discriminated results, whether this cause ethical problems in AI application. Thanks!

[sdbaier]

Dear Professor Wachter, in Bias Preservation in Machine Learning: The Legality of Fairness Metrics Under EU Non-Discrimination Law, you mention that bias preserving metrics often perpetuate social injustices and discriminatory effects – given that algorithms are trained on potentially flawed historical data (falsely treating it as neutral), problematic data collection or sampling procedures, as well as in itself potentially biased depending on their objective function, this makes sense. Logically, bias transforming metrics should be preferred.

On page 777, you mention “[…] for use cases where existing biases are normatively acceptable, bias preserving metrics may be preferable. […] Ideally, users should test as broadly as possible with both bias preserving and transforming metrics to investigate the fairness of their decision-making systems.”

Even if biases are acceptable, why would bias preserving metrics trump bias transforming metrics? Would this not reinforce the issue of perpetuating biases through relying on bias preserving metrics?

[yhchou0904]

Hi Professor Wachter, thank you for sharing your fantastic idea with us! The disparity/inequality related bias is one of the most important topics, and I am always interested in exploring the solutions given by the scholars. At the same time, I am always wondering the trade-off between solving model's discrimination problem and the performance. It seems that we are not always being lucky to have a better label or metric that could help us lower the bias and improve/keep the performance at the same time. As a government or administrative department, is there a more proper way or rule of thumb to think of how to balance the technology progress and inequality especially in this area?

[MkramerPsych]

Dr. Wachter,

Thank you for sharing your research with us! My question arguably borders closer to philosophy than data science, but I would be interested in hearing your response nonetheless. Given your argument that bias mitigation strategies in ML are grounded in statistical methods to reduce quantifiable technical biases rather than the root cause of inequality, I am curious about how one would defend against the argument that "algorithms aren't biased, data are". I am particularly interested in the legal and ethical obligations of software developers as compared to data collection personnel, as I have seen numerous examples where ML engineers essentially reduce bias to features of training data.

[zhiqianc]

Hi Professor Wachter, thanks for sharing the excellent research with us! I am so impressed by the definition of algorithm fairness. In your article, you proposed a system based on EU laws. I am wondering if there are some other applications for this system. In addition, if the system can be used all over the world.

[fiofiofiona]

Hi Professor Wachter, thank you for sharing your work. You mentioned that more works of ML on non-discrimination law have been done on US laws than EU laws. I wonder how widely and how likely you envision that these machine learning models can be customized and applied to other countries?

[yierrr]

Thank you so much for sharing! I am interested about how this can be related to systemic discrimination as well.

[jiehanL]

Hi Professor Wachter, thank you for sharing your work! Given that the development of AI-based solutions is a very important goal for both companies and some governments, the problem of lack of fairness seems a bit not sufficient or efficient enough to be solved on a case-by-case basis? Do you think current regulation/law needs to be updated as a whole, and how should the regulation/law pertains to transparency co-exist and interact with intellectual property laws?