search

privacytrustlab / ml_privacy_meter

Privacy Meter: An open-source library to audit data privacy in statistical and machine learning algorithms.
MIT License
588 stars 100 forks source link

AttributeError: 'attack_data' object has no attribute 'means' #19

Closed gongzhimin closed 3 years ago

gongzhimin commented 3 years ago

Hello, there might be a trivial flaw in your code.

In line 39 of the file ml_privacy_meter/tutorials/attack_alexnet.py: image it is written that the means and standard deviations for normalization will be calculated if unset.

I understand that you expect this to be determined by this line in ml_privacy_meter/tutorials/attack_alexnet.py: image

However, it doesn't work but instead introduces an error on my machine: AttributeError: 'attack_data' object has no attribute 'means'

The fact that you didn't declare these two attributes (means and stddevs) in the initialization method of class attack_data is presumably the reason. So the bug crawled into your program. image

Looking forward to your reply.

mihirkhandekar commented 3 years ago

Thanks for pointing out this issue. We will test it and fix the issue in the code accordingly.

gongzhimin commented 3 years ago

Hello, more information provided to help you: The error shot me after this line of code was commented out. image Since I hope your tool can help to calculate the means and standard deviations, instead of assign them like hyperparameters.

And I have a question: is your program an implementation of supervised training on shadow models? Namely, is model A a shadow model of model B? image

As the code shows me, it's assumed that a subset of the training set is known to the attacker image as well as some data from the same underlying distribution that is not contained in the training set. image

image

Do I get it right? I'm sorry for my lack of expression.

mihirkhandekar commented 3 years ago

Thank you for providing the additional information. Model A can be used as a shadow model to evaluate the performance of your membership inference model on Model B. Thank you for the diagram as well, it is a good representation of the data. However, I would like to point out that the non-member set does not have an intersection with the cifar100_train.txt.npy, since this file is only generated using the training data (Member set).

gongzhimin commented 3 years ago

I get that. It's my mistake. image And thank you for your reply (^_^)