dashaasienga
commented
1 year ago Thanks for sharing this! I went through it and it's definitely very useful for my thesis! I think it lay a really good framework/ foundation for how my thesis could look like in the end. Essentially, they created 2 new loss functions that would attempt to solve or mitigate the issue of induced bias. Induced bias refers to a situation where we've dropped the protected attributes such as race or gender, but we've left other attributes highly correlated with these protected attributes and can serve as proxies, essentially introducing new bias to the model. They used simulations to generate synthetic data sets that they tested their own loss functions with. They also introduced bias into real world data sets to perform comparisons with current state-of-the-art fairness ML models. They used the AIF360 toolkit I mentioned to you last week, which is open-source, so that should be a great resource when it comes to model comparisons!
It was also a great introduction to the notation used in this field and areas that I could look further into. Based on data sets, I'm thinking COMPAS is one I'd love to work with!
They suggested a potential for future work which would involve relying on realistic simulations of discriminations and test whether a given learning method is able to retrieve the non-discriminatory data-generating process. They also suggested testing the proposed methods on more complex non-linear models. These are 2 areas I'd love to discuss with them as they provide an avenue for something we could build on.
The notation was also a bit heavy for me, so I'm thinking a good place to invest my time in for the next couple of weeks is understanding the current fairness metrics and their probabilistic definitions. This will allow me to understand how exactly we can assess fairness of a method using existing statistical definitions and come handy down the line when I'm doing my own evaluations.
Other questions I have for them are:
- How do they determine that the outcome is not influenced by the protected attribute, even if in an indirect way?
- How did they measure the influence of features?
- What was the method of implementation and testing?
katcorr
I added this article to the repo. I have not read it, so it may not be useful at all for you / your thesis! But (1) the three authors are all from UMass, so if something does seem relevant/interesting to you, perhaps we can follow-up with them; and (2) the one thing I did notice upon scanning is that it refers to three datasets that seem to be publicly available, so maybe are options for you as well (for down the line . . .)