search

privML / privacy-evaluator

The privML Privacy Evaluator is a tool that assesses ML model's levels of privacy by running different attacks on it.
MIT License
17 stars 17 forks source link

Implement MIA #8

Closed budmil closed 3 years ago

budmil commented 3 years ago

Parameters: model, training data and dict with keywords to perform attack

User membership inference attack from framework with defined (default)parameters

Acceptance Criteria

Be able to perform example with correct/default parameters

blauertee commented 3 years ago

Consider #12 before you start coding :)

blauertee commented 3 years ago

I don't know weather I'm late to the party by finding that out, but I did not understand it thus far and stumbled upon it doing my research for #42. So I thought, I might just share it here. The BlackBox attack is called black box attack, because it trains a model (the black box) that predicts weather data points or chunks of data have been part of the training data set (weather they're a member). This is why in the examples training is done. Not to train the model that will be attacked. But to train a model that can predict membership on other models, the attack model. I'm not sure weather this type of membership inference attack makes sense in our use case, maybe we should just go with the rule based approach.

Friedrich-Mueller commented 3 years ago

Black Box refers to an unknown type of model. Nothing about structure or meta-parameters is know. See following quote from the paper: We assume that the attacker has query access to the model and can obtain the model’s prediction vector on any data record. The attacker knows the format of the inputs and outputs of the model, including their number and the range of values they can take. We also assume that the attacker either (1) knows the type and architecture of the machine learning model, as well as the training algorithm, or (2) has black-box access to a machine learning oracle (e.g., a “machine learning as a service” platform) that was used to train the model. In the latter case, the attacker does not know a priori the model’s structure or meta-parameters.

marisanest commented 3 years ago

Solved by PR #51