dssg / aequitas

Bias Auditing & Fair ML Toolkit
http://www.datasciencepublicpolicy.org/aequitas/
MIT License
694 stars 114 forks source link

Confusion of Statistical Parity and Impact Parity? #89

Open LiFaytheGoblin opened 4 years ago

LiFaytheGoblin commented 4 years ago

Here: https://github.com/dssg/aequitas/blob/master/docs/source/examples/compas_demo.ipynb Under "How do I assess model fairness?" > "Pairities Calcuated" it sais

Predicted Positive Ratio k Parity Statistical Parity

According to your definitions Predicted Positive Ratio k is PPR and Predicted Positive Ratio g is PPrev. So Aequitas defines Statistical Parity as PPR Parity That is not the definition according to Verma & Rubin 2018, who say Statistical Parity was PPrev Parity. Then again, if taking Statistical Parity synonymous to Demographic Parity, your definition of it being PPR Parity makes more sense I think.

So, any thoughts on this?