This code replicates the experiments from the following paper:
Pang Wei Koh, Jacob Steinhardt, and Percy Liang
Stronger Data Poisoning Attacks Break Data Sanitization Defenses
We use the following dependencies:
For historical reasons, this codebase is split into Python and MATLAB components:
matlab/
folder.You can download the datasets we used here. For convenience, we have included .npz and .mat versions of the datasets for use in the Python and MATLAB files.
To set up the directory structure, you might want to edit the following files:
datasets.py
Set DATA_FOLDER to where the datasets/decoy parameters are stored and OUTPUT_FOLDER to where you'd like the attacks to be output. This affects all of the Python files and is independent of the MATLAB files.
pathdef.m
Needed in order to find gurobi, yalmip, sedumi, and the sever/svm subfolder. Will likely need to re-generate based on your local installation.
GRB_LICENSE_FILE
If you're using gurobi, this environmental variable needs to be set in your shell.
run_gradient_em_attack.py
To run the influence attack for a dataset using the same settings as we used in the paper, execute python run_gradient_em_attack.py --em_iter 0 --total_grad_iter 10000 --dataset enron --use_slab --epsilon 0.03
, replacing 'enron' with the names of other datasets if needed. em_iter controls the number of iterative updates to the feasible set; we set this to 5000 to generate Figure 5.
copy_over_gradient_attacks.py
This is a helper file that converts the influence attack files into a similar format as the other attacks below.
Execute, e.g., python copy_over_gradient_attacks.py influence-em-0
or python copy_over_gradient_attacks.py influence-em-5000
after the attack is finished.
influence/
Git repo for "Understanding Black-box Predictions via Influence Functions". We use helper functions from this repo to calculate influence.
generateThetaFnc.m
Generates target parameters, which are needed to run the KKT and min-max attacks. Calls generateTheta with the appropriate arguments. Uses pruneTheta to prune the set of generated theta down to a manageable amount.
run_kkt_attack.py
To run the kkt attack for a dataset using the same settings as we used in the paper, execute python run_kkt_attack.py kkt-loss-slab --dataset enron --repeat_points 2
, replacing 'enron' with the names of other datasets if needed.
generateAttackFnc.m
Runs the min-max attack for a dataset using the same settings as we used in the paper. Assumes that generateThetaFnc has already been run with results saved to the locations specified in target_file. Calls generateAttackAll, which calls generateAttackTar (this final file is where most of the work of actually generating an attack occurs).
sever/
Git repo for "Sever: A Robust Meta-Algorithm for Stochastic Optimization".
We make use of several helper functions for evaluating defenses and training models.
Other files:
prettyPrint.m
Helper file to pretty-print a struct.
randRound.m
Helper file to randomly round a real-valued vector to an integer-valued vector.
test_defenses_for_all_epsilons.py
To evaluate an attack after generating it, run python test_defenses_for_all_epsilons.py enron [attack_name] [--epsilon 0.03]
.
attack_name can be, for example, 'influence-em-0', 'kkt-loss-slab-repeat2', or 'mm-loss-slab-train'.