Closed Erdnaf closed 3 years ago
@fraboeni We are supposed to calculate the attacker's advantage for every slice precisely like TF privacy is doing, right? Not the membership privacy risk score? So the two are not really related, if I understand correctly, because we're a little confused here.
Here is the code for calculation: https://github.com/jtorhoff/privacy/blob/master/tensorflow_privacy/privacy/privacy_tests/membership_inference_attack/data_structures.py#L411
Slicing correctly classified vs wrongly classified
Slicing per class
Output: should give attack advantage score for the whole dataset, and for each slice individually
Have slices as optional arguments (if nothing is specified when running the membership inference attack, then run it on whole dataset). It should also be possible to specify several in one time, which would run attack on every slice
sidenote: run tensorflow's attack and learn from it!