search

Trusted-AI / AIF360

A comprehensive set of fairness metrics for datasets and machine learning models, explanations for these metrics, and algorithms to mitigate bias in datasets and models.
https://aif360.res.ibm.com/
Apache License 2.0
2.46k stars 840 forks source link

MetaFairClassifier not working with certain dataset loaders #454

Open richeekAbacus opened 1 year ago

richeekAbacus commented 1 year ago

The example notebook https://github.com/Trusted-AI/AIF360/blob/master/examples/demo_meta_classifier.ipynb fails to run the MetaFairClassifier successfully.

I get errors like:

/opt/conda/envs/env/lib/python3.9/site-packages/aif360/algorithms/inprocessing/celisMeta/FalseDiscovery.py:31: RuntimeWarning: invalid value encountered in divide
  prob_y_1 = (prob_1_1 + prob_1_0) / total
/opt/conda/envs/env/lib/python3.9/site-packages/aif360/algorithms/inprocessing/celisMeta/FalseDiscovery.py:32: RuntimeWarning: invalid value encountered in divide
  prob_z_0 = (prob_m1_0 + prob_1_0) / total
/opt/conda/envs/env/lib/python3.9/site-packages/aif360/algorithms/inprocessing/celisMeta/FalseDiscovery.py:33: RuntimeWarning: invalid value encountered in divide
  prob_z_1 = (prob_m1_1 + prob_1_1) / total
/opt/conda/envs/env/lib/python3.9/site-packages/aif360/algorithms/inprocessing/celisMeta/FalseDiscovery.py:35: RuntimeWarning: invalid value encountered in divide
  probc_m1_0 = prob_m1_0 / total
/opt/conda/envs/env/lib/python3.9/site-packages/aif360/algorithms/inprocessing/celisMeta/FalseDiscovery.py:36: RuntimeWarning: invalid value encountered in divide
  probc_m1_1 = prob_m1_1 / total
Traceback (most recent call last):
  File "/home/richeek/NAS-Fairness-Tabular/baselines.py", line 333, in <module>
    main()
  File "/home/richeek/NAS-Fairness-Tabular/baselines.py", line 209, in main
    debias_model = MetaFairClassifier(tau=0.9, sensitive_attr=args.privilege_mode,
  File "/opt/conda/envs/env/lib/python3.9/site-packages/aif360/algorithms/transformer.py", line 27, in wrapper
    new_dataset = func(self, *args, **kwargs)
  File "/opt/conda/envs/env/lib/python3.9/site-packages/aif360/algorithms/inprocessing/meta_fair_classifier.py", line 66, in fit
    self.model = self.obj.getModel(self.tau, x_train, y_train,
  File "/opt/conda/envs/env/lib/python3.9/site-packages/aif360/algorithms/inprocessing/celisMeta/General.py", line 99, in getModel
    t = self.getValueForX(dist, a, b, params, z_1, X)
  File "/opt/conda/envs/env/lib/python3.9/site-packages/aif360/algorithms/inprocessing/celisMeta/FalseDiscovery.py", line 18, in getValueForX
    u_1, u_2, l_1, l_2 = params
TypeError: cannot unpack non-iterable NoneType object

I feel this is a dataset related issue, but I can't figure out why this is happening

serbanstan commented 1 year ago

Downgrading to scipy-1.5.4 may be a temporary fix. The problem comes from the computation of multivariate_normal.pdf, which in the newer version of scipy returns very small values compared to the older versions.

boghrati commented 10 months ago

Hi, I encounter the same issue with metafair classifier as well. My code was correct about a year ago, but now, it raises an error. I tried downgrading scipy, but it didn't help. I was wondering if there is any solution for this error? TypeError: cannot unpack non-iterable NoneType object

team-daniel commented 7 months ago

Downgrading to scipy-1.5.4 may be a temporary fix. The problem comes from the computation of multivariate_normal.pdf, which in the newer version of scipy returns very small values compared to the older versions.

You have just saved hours of heartache 🥰 thank you so much! I downgraded to scipy==1.8.1 if anyone in the future sees this!