To make analysis of fairness audit results a little more user-friendly, we should add an example notebook illustrating how to use aequitas's visualization tools both to audit a single model for fairness as well as look at trade-offs between fairness and accuracy metrics (e.g., via a scatterplot of precision@k vs recall disparity across model groups at a given train end time). Doing so will also require adding a simple utility function to load the aequitas outputs in the database into the necessary data types in python for the analysis.
I think a reasonable place for this to live would probably be in a new fairness folder in /src/triage/component/postmodeling/.
To make analysis of fairness audit results a little more user-friendly, we should add an example notebook illustrating how to use aequitas's visualization tools both to audit a single model for fairness as well as look at trade-offs between fairness and accuracy metrics (e.g., via a scatterplot of precision@k vs recall disparity across model groups at a given train end time). Doing so will also require adding a simple utility function to load the aequitas outputs in the database into the necessary data types in python for the analysis.
I think a reasonable place for this to live would probably be in a new
fairnessfolder in/src/triage/component/postmodeling/.