Implement slicing as in Tensorflow Membership privacy attack

[Erdnaf]

• Slicing correctly classified vs wrongly classified

• Slicing per class

• Output: should give attack advantage score for the whole dataset, and for each slice individually

• Have slices as optional arguments (if nothing is specified when running the membership inference attack, then run it on whole dataset). It should also be possible to specify several in one time, which would run attack on every slice

• sidenote: run tensorflow's attack and learn from it!

[blauertee]

Blogpost by Franziska explainging MIAs with tf privacy

[marisanest]

Helpful links:

• TensorFlow notebook with examples of the use of the membership_probability (i.e. the privacy risk score)
• TensorFlow notebook with examples of the use of MIAs (i.e. the privacy risk score)

[jtorhoff]

@fraboeni We are supposed to calculate the attacker's advantage for every slice precisely like TF privacy is doing, right? Not the membership privacy risk score? So the two are not really related, if I understand correctly, because we're a little confused here.

Here is the code for calculation: https://github.com/jtorhoff/privacy/blob/master/tensorflow_privacy/privacy/privacy_tests/membership_inference_attack/data_structures.py#L411