Closed minaremeli closed 2 years ago
Hi @minaremeli Thank you very much for proposing this issue and planning to contribute to ART! I think this will be a very nice tool for inference evaluations. You can target branch dev_1.8.0
with your code development. Please let us know anytime if you have questions!
@abigailgold This project is probably of interest for your work.
@minaremeli I'd like to introduce to you @abigailgold who is developing new inference attacks and has implemented most of the current inference attacks in ART.
I think this is great, and agree with the approach of adding an additional calibrate method. Please make sure that it is clear from the documentation that only one of them needs to be called. Thanks!
Hi @beat-buesser @abigailgold!
I have implemented the attack as discussed and tried it out locally (demo notebook). You can look at the branch I worked on here if you would like to take a peek.
However I might need some help with the unit tests. Currently, only the infer
function is tested. My guess is that this is because the calibrate_distance_threshold
function is slow. The same is true for the new calibrate_distance_threshold_unsupervised
function. What is to be done here?
Hi @minaremeli Actually there are a few tests that do run calibrate_distance_threshold, for example look at: test_label_only_boundary_distance_prob_calib in https://github.com/Trusted-AI/adversarial-robustness-toolbox/blob/main/tests/attacks/inference/membership_inference/test_label_only_boundary_distance.py (line 54). It just needs to be run with specific parameters that limit the runtime (for example very low max_iter and max_eval values). You could try to do the same for the new method.
Hi @minaremeli This is a very nice notebook and implementation of the new unsupervised calibration method! If you like you can add the notebook to the notebooks of ART, I'm sure it will be helpful for many users of ART!
I have forgotten to mention it before, but one of the checks running on pull request is DCO which requires that all commits have been signed with the git -S
option. If you open the pull request, it will show how to sign existing unsigned commits, which works well if there are no merge commits in between.
Is your feature request related to a problem? Please describe. Would be interesting to compare to Choquette-Choo et al.'s work which is already implemented (
LabelOnlyDecisionBoundary
).Describe the solution you'd like Both attacks are basically the same, except for how they calibrate the distance threshold tau (which is used to infer membership). Therefore I propose to extend the existing
LabelOnlyDecisionBoundary
with a new threshold calibrating function. Thus we will have two functions besidesinfer(x,y)
:calibrate_distance_threshold(x_train, y_train, x_test, y_test)
- Choquette-Choo et al. distance threshold calibration (already implemented)calibrate_distance_threshold_unsupervised(top_t, num_samples, max_queries)
- Li and Zhang distance threshold calibrationunsupervised
because it requires no data to set the distance thresholdtop_t
: top-t percentilenum_samples
: number of random samples to generatemax_queries
: maximum number of queriesDescribe alternatives you've considered I have considered inheritance from
LabelOnlyDecisionBoundary
, however this would require me to either:calibrate_distance_threshold()
in the child class with the parameters shown above (which if I know correctly is bad practice)init()
, and callcalibrate_distance_threshold()
with no passed parameters (would require change of signature in parent method with default values set toNone
+ parameter check in parent so it doesn't get called withNone
)Additional context Paper link: https://arxiv.org/abs/2007.15528