OperatorFairness (Issue #9)

[stefan-grafberger]

Issue #, if available: #9

Description of changes:

• Score DagNode is now being represented as predict and score
• Added support for estimator.predict
• Added support for sklearn accuracy_score, fairlearn MetricFrame and fairlearn equalized_odds_difference
• Added support for GridSearchCV wrapping estimators
• Modified healthcare.py to use all these
• Updated DataCorruption what-if analysis to work with multiple score nodes
• Added OperatorFairness what-if analysis
  • Can only analyse transformer and selection nodes for now
  • Project_modify is a bit more difficult, changes of crashes is higher and we often need to ensure we make the same modifications on train and test set
  • No optimisation yet
• Added a short section in the demo notebook for OperatorFairness

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[stefan-grafberger]

Codecov Report

Merging #11 (75340ef) into main (e840731) will increase coverage by 0.38%. The diff coverage is 98.09%.

@@            Coverage Diff             @@
##             main      #11      +/-   ##
==========================================
+ Coverage   96.05%   96.43%   +0.38%     
==========================================
  Files          23       25       +2     
  Lines        2357     2889     +532     
==========================================
+ Hits         2264     2786     +522     
- Misses         93      103      +10     

Impacted Files	Coverage Δ
mlwhatif/analysis/_data_corruption.py	96.35% <93.18%> (-0.35%)	:arrow_down:
mlwhatif/analysis/_operator_fairness.py	97.35% <97.35%> (ø)
mlwhatif/monkeypatching/_patch_sklearn.py	98.96% <98.38%> (-0.42%)	:arrow_down:
mlwhatif/analysis/_analysis_utils.py	100.00% <100.00%> (ø)
mlwhatif/execution/_multi_query_optimizer.py	100.00% <100.00%> (ø)
mlwhatif/execution/_stat_tracking.py	92.85% <100.00%> (+0.07%)	:arrow_up:
mlwhatif/instrumentation/_operator_types.py	90.00% <100.00%> (+0.25%)	:arrow_up:
mlwhatif/monkeypatching/_patch_fairlearn.py	100.00% <100.00%> (ø)
mlwhatif/monkeypatching/_patch_pandas.py	90.13% <100.00%> (+0.42%)	:arrow_up:
mlwhatif/analysis/_what_if_analysis.py	78.94% <0.00%> (+5.26%)	:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us.