search

Trusted-AI / AIF360

A comprehensive set of fairness metrics for datasets and machine learning models, explanations for these metrics, and algorithms to mitigate bias in datasets and models.
https://aif360.res.ibm.com/
Apache License 2.0
2.4k stars 827 forks source link

Include additional classification metrics #88

Open nrkarthikeyan opened 5 years ago

nrkarthikeyan commented 5 years ago

Include the following metrics:

  1. Equalized odds difference: max(|FPR_unpriv - FPR_priv|, |TPR_unpriv - TPR_priv|)
  2. Generalized equalized odds difference: max(|GFPR_unpriv - GFPR_priv|, |GTPR_unpriv - GTPR_priv|)
  3. Generalized selection rate: mean score possibly conditioned by the group E[\hat{S}]
nrkarthikeyan commented 1 year ago

This can be implemented for the "classic": https://github.com/Trusted-AI/AIF360/blob/master/aif360/metrics/classification_metric.py

as well as the sklearn-compatible: https://github.com/Trusted-AI/AIF360/blob/master/aif360/sklearn/metrics/metrics.py

versions

divyagaddipati commented 11 months ago

Hi! Working on this issue. Can I get assigned to this?

divyagaddipati commented 11 months ago

what does "generalized selection rate" mean? I couldn't find much about it anywhere

hoffmansc commented 10 months ago

Status:

Classic:

sklearn:

emilymacq commented 10 months ago

Hi! This PR addresses the equalized odds difference and generalized equalized odds difference for sklearn.