search

alan-turing-institute / tapas

MIT License
31 stars 14 forks source link

Threat model: proposition and implementation #10

Closed fhoussiau closed 2 years ago

fhoussiau commented 2 years ago

In this branch, I have written a first draft of a "threat model" class (in threat_models/base_classes.py and threat_models/mia.py). This class is composed of two main parts:

  1. All the information available to an attacker (e.g., auxiliary dataset, black-box access to SDG, target records). This can also be used to define how to generate training datasets -- although an attack does not necessarily need to use it.
  2. The logic to generate testing datasets to evaluate the attack.

(this should probably be explained somewhere, as it's maybe not intuitive, I've put a longish description in threat_models/base_classes.py but it might be worth putting it in longer form somewhere?)

I have modified attacks/groundhog.py and run_groundhog.py to use this class.

Could I have your input on this class? (and the name, the more I think of it the less I like "threat model").

JamesJordon commented 2 years ago

@fhoussiau and I had a back and forth yesterday over slack (my bad for not seeing this issue). I'll try to summarise what I landed on in my thinking for the ThreatModel class.

  1. It should house the adversary's background knowledge (i.e. should have an attribute .adv_data and .shadow_model, indicating. I think .adv_data should be a dictionary containing auxiliary data, and any of the real data it has access to (I've implemented this in this branch, in the threat_models/mia.py file). .shadow_model can either be the same as the generator, or different, in line with the idea that we investigate what happens when an adversary has imperfect knowledge of the generative mechanism.
  2. It should house the logic for testing, not necessarily creating test datasets. I think this should be implemented via a .test(attack) method. I don't feel strongly either way about whether the report should be part of this, or whether test should just output the raw outputs of the attack.
  3. We should rename the class to something more appropriate, but we haven't thought of anything concrete.

I actually think 2. and 3. could be separate classes, but let's implement it as one for now.

I also feel that target should not be saved in ThreatModel.