valmik-patel
commented
7 months ago https://github.com/dssg/aequitas/pull/175
I created this draft PR to add testing to the library using pytest. Currently, the script will run tests defined in the tests folder and output a coverage report. This script can be easily added to GitHub actions to ensure that all tests are passing before a PR is merged.
PS: I had to update the version of one of the libraries because the tests were failing on the previous version.
Current coverage report:
---------- coverage: platform darwin, python 3.9.16-final-0 ----------
Name Stmts Miss Cover
-----------------------------------------------------------------------------------------------
src/aequitas/__init__.py 3 0 100%
src/aequitas/audit.py 108 84 22%
src/aequitas/bias.py 263 201 24%
src/aequitas/fairness.py 104 22 79%
src/aequitas/flow/__init__.py 2 2 0%
src/aequitas/flow/datasets/__init__.py 4 4 0%
src/aequitas/flow/datasets/baf.py 107 107 0%
src/aequitas/flow/datasets/dataset.py 50 50 0%
src/aequitas/flow/datasets/folktables.py 127 127 0%
src/aequitas/flow/datasets/generic.py 100 100 0%
src/aequitas/flow/evaluation/__init__.py 8 8 0%
src/aequitas/flow/evaluation/fairness.py 22 22 0%
src/aequitas/flow/evaluation/performance.py 9 9 0%
src/aequitas/flow/experiment/__init__.py 3 3 0%
src/aequitas/flow/experiment/_configs.py 25 25 0%
src/aequitas/flow/experiment/default.py 39 39 0%
src/aequitas/flow/experiment/experiment.py 155 155 0%
src/aequitas/flow/methods/__init__.py 4 4 0%
src/aequitas/flow/methods/base_estimator/__init__.py 5 5 0%
src/aequitas/flow/methods/base_estimator/base_estimator.py 18 18 0%
src/aequitas/flow/methods/base_estimator/lightgbm.py 26 26 0%
src/aequitas/flow/methods/base_estimator/logistic_regression.py 37 37 0%
src/aequitas/flow/methods/base_estimator/neural_network.py 84 84 0%
src/aequitas/flow/methods/base_estimator/random_forest.py 37 37 0%
src/aequitas/flow/methods/inprocessing/__init__.py 3 3 0%
src/aequitas/flow/methods/inprocessing/fairgbm.py 16 16 0%
src/aequitas/flow/methods/inprocessing/fairlearn_classifier.py 53 53 0%
src/aequitas/flow/methods/inprocessing/inprocessing.py 10 10 0%
src/aequitas/flow/methods/postprocessing/__init__.py 5 5 0%
src/aequitas/flow/methods/postprocessing/balanced_group_threshold.py 67 67 0%
src/aequitas/flow/methods/postprocessing/group_threshold.py 23 23 0%
src/aequitas/flow/methods/postprocessing/identity.py 10 10 0%
src/aequitas/flow/methods/postprocessing/postprocessing.py 10 10 0%
src/aequitas/flow/methods/postprocessing/threshold.py 50 50 0%
src/aequitas/flow/methods/preprocessing/__init__.py 3 3 0%
src/aequitas/flow/methods/preprocessing/correlation_suppression.py 48 48 0%
src/aequitas/flow/methods/preprocessing/data_repairer.py 43 43 0%
src/aequitas/flow/methods/preprocessing/feature_importance_suppression.py 48 48 0%
src/aequitas/flow/methods/preprocessing/identity.py 12 12 0%
src/aequitas/flow/methods/preprocessing/label_flipping.py 104 104 0%
src/aequitas/flow/methods/preprocessing/massaging.py 45 45 0%
src/aequitas/flow/methods/preprocessing/preprocessing.py 16 16 0%
src/aequitas/flow/methods/preprocessing/prevalence_sample.py 81 81 0%
src/aequitas/flow/optimization/__init__.py 157 157 0%
src/aequitas/flow/plots/__init__.py 0 0 100%
src/aequitas/flow/plots/bootstrap/__init__.py 2 2 0%
src/aequitas/flow/plots/bootstrap/plot.py 42 42 0%
src/aequitas/flow/plots/bootstrap/visualize.py 48 48 0%
src/aequitas/flow/plots/pareto/__init__.py 2 2 0%
src/aequitas/flow/plots/pareto/plot.py 125 125 0%
src/aequitas/flow/plots/pareto/visualize.py 62 62 0%
src/aequitas/flow/utils/__init__.py 6 6 0%
src/aequitas/flow/utils/colab.py 33 33 0%
src/aequitas/flow/utils/config.py 28 28 0%
src/aequitas/flow/utils/evaluation.py 88 88 0%
src/aequitas/flow/utils/imports.py 16 16 0%
src/aequitas/flow/utils/labeled_frame.py 19 19 0%
src/aequitas/flow/utils/logging.py 59 59 0%
src/aequitas/flow/utils/metrics.py 6 6 0%
src/aequitas/flow/utils/results.py 45 45 0%
src/aequitas/group.py 127 44 65%
src/aequitas/plot/__init__.py 5 0 100%
src/aequitas/plot/bubble_concatenation_chart.py 26 12 54%
src/aequitas/plot/bubble_disparity_chart.py 144 117 19%
src/aequitas/plot/bubble_metric_chart.py 119 95 20%
src/aequitas/plot/commons/__init__.py 0 0 100%
src/aequitas/plot/commons/helpers.py 21 12 43%
src/aequitas/plot/commons/initializers.py 76 64 16%
src/aequitas/plot/commons/labels.py 7 0 100%
src/aequitas/plot/commons/legend.py 27 18 33%
src/aequitas/plot/commons/metrics.py 2 0 100%
src/aequitas/plot/commons/scales.py 33 25 24%
src/aequitas/plot/commons/style/__init__.py 0 0 100%
src/aequitas/plot/commons/style/classes.py 86 0 100%
src/aequitas/plot/commons/style/color.py 19 0 100%
src/aequitas/plot/commons/style/sizes.py 39 0 100%
src/aequitas/plot/commons/style/text.py 8 0 100%
src/aequitas/plot/commons/tooltips.py 18 14 22%
src/aequitas/plot/commons/validators.py 69 57 17%
src/aequitas/plot/summary_chart.py 131 104 21%
src/aequitas/plot/xy_metrics_chart.py 91 73 20%
src/aequitas/plotting.py 653 598 8%
src/aequitas/preprocessing.py 50 23 54%
src/aequitas/squarify_flipped.py 119 102 14%
src/aequitas/version.py 1 0 100%
-----------------------------------------------------------------------------------------------
TOTAL 4596 3912 15%
The aim of this issue is to develop a robust and effective unit and integration testing strategy.
We need to: