search

tensorflow / privacy

Library for training machine learning models with privacy for training data
Apache License 2.0
1.95k stars 452 forks source link

DP accountant and composition #186

Open XRC99 opened 2 years ago

XRC99 commented 2 years ago

Hi,

I read some of paper (both DP and RDP) such as [1,2], they mentioned that the composition of epsilon is O(\sqrt(T)) when all the iterations have the same epsilon (homogenous mechanisms). I think that in DPSGD, each iteration has the same epsilon. But the epsilon I got from compute_dp_sgd_privacy() didn't satisfy the O(\sqrt(T)) composition when I changed the number of epoches. I am wondering why the composition results and accountant results are not the same.

Thank you in advance!!!

[1]Abadi, Martin, et al. "Deep learning with differential privacy." Proceedings of the 2016 ACM SIGSAC conference on computer and communications security. 2016. [2]Mironov, Ilya. "Rényi differential privacy." 2017 IEEE 30th Computer Security Foundations Symposium (CSF). IEEE, 2017.

galenmandrew commented 2 years ago

There are several papers cited in the accounting code that are refinements of the works you cited. Probably what you are observing is from those refined bounds.