Why are we enforcing fairness over the training data?

[Innixma]

For example in https://colab.research.google.com/drive/1GKzlciJ-Yy3Cej7dJmzK9YS6k12p8fox?usp=sharing

# Modify predictor to enforce fairness over the train_data with respect to groups given by the column 'sex'
fpredictor = FairPredictor(predictor,train_data,'sex')

Train data accuracy is irrelevant for how fair the model is on test-time data.

You should either pass the validation data or holdout data at this point.

[ChrisMRuss]

There's a discussion of the issue in best practices. https://github.com/autogluon/autogluon-fair#best-practices

Essentially it's a trade-off. For rare groups we would need a large validation set to reliably compute error rates -- hold-out is generally too small. For small tabular data, autogluon is robust enough against overfitting that, at least on adult, using train also works.

Either approach is viable for gender, but for ethnicity there's not enough data and you benefit from using the entirety of train.

[Innixma]

I'd push back here: Random Forest literally memorizes the training data. Similar with KNN. This is so severe that any insights you get on training data performance will be largely useless.

We can do much better using the validation data, but an ever better idea is to use the out-of-fold prediction probabilities when bagging. This is akin to using the training data, but without the overfitting problem. This would work very well, I am quite certain.

I am pretty sure I can add this in a few lines of code, and it will make a big difference. I can try to allocate bandwidth, although can't promise I will find time.

[ChrisMRuss]

That sounds awesome.

[Innixma]

18 implements this