Closed EliottZemour closed 2 years ago
Thanks a lot for opening this issue, Elliott! I'll look into it -- provisionally marking as #bug.
Hi Eliott! As I mentioned to Bogdan in private msg, the issue seems that you're using min-estimate
as opposed to last-estimate
.
The difference is the following. fbleau
estimates the risk (on a fixed test set) for various sizes of the training set. Clearly, this estimate will have some variability, even in the "random data" case that you consider; (imagine outputting predictions uniformly at random: you wouldn't get exactly 50% error).
By selecting min-estimate
, you're taking the best error observed for some training set size. In fbleau
, I recommend this for evaluating the strength of a defence against an attacker: we'd rather overestimate their power than underestimate it.
However, what you want is to assert that fbleau
is not biased; in this case, you need to use last-estimate
, which represents the estimate obtained by training on the entire training set.
I will make sure that the documentation reflects this. Thanks a lot for rising this issue, and let me know if you encounter any other problem.
I'm using fbleau on log-losses features for membership inference attacks (mia), and I noticed that fbleau consistently beats random guessing on control models: constant (losses = np.ones) and random.
Here's the gist to reproduce the results with python run_fbleau: I obtain attackers accuracy > 0.5 on most of the control models: https://gist.github.com/EliottZemour/2ed30b875be5cfe427bc44b55ec74f01