tensorflow / privacy

Library for training machine learning models with privacy for training data
Apache License 2.0
1.92k stars 447 forks source link

Insecure Random Number Generator #349

Open gonzalo-munillag opened 1 year ago

gonzalo-munillag commented 1 year ago

Hello,

I would like to bring to your attention that using the random number generator from TensorFlow could lead to vulnerabilities when sampling from a distribution to fulfill differential privacy during training: https://www.tmlt.io/research/tiny-bits-matter-precision-based-attacks-on-differential-privacy

PyTorch Opacus uses a secure RNG: https://opacus.ai/api/privacy_engine.html

In contrast, TensorFlow RNG: https://www.tensorflow.org/api_docs/python/tf/random/Generator https://stackoverflow.com/questions/63350248/is-tf-random-normal-cryptographically-secure

Additionally, there is no documentation that states the use of floating-point vulnerability protection as in https://scholar.google.com/citations?view_op=view_citation&hl=en&user=hg3A9TgAAAAJ&citation_for_view=hg3A9TgAAAAJ:dhFuZR0502QC and https://research.ibm.com/publications/secure-random-sampling-in-differential-privacy

Kind regards, Gonzalo

kairouzp commented 1 year ago

Thanks so much for pointing this out. We are aware of this issue (see Section 8 of https://arxiv.org/pdf/1812.06210.pdf).

On Mon, Nov 7, 2022 at 2:50 PM Gonzalo Munilla Garrido < @.***> wrote:

Hello,

I would like to bring to your attention that using the random number generator from TensorFlow could lead to vulnerabilities when sampling from a distribution to fulfill differential privacy during training: https://www.tmlt.io/research/tiny-bits-matter-precision-based-attacks-on-differential-privacy

PyTorch Opacus uses a secure RNG: https://opacus.ai/api/privacy_engine.html

In contrast, TensorFlow RNG: https://www.tensorflow.org/api_docs/python/tf/random/Generator

https://stackoverflow.com/questions/63350248/is-tf-random-normal-cryptographically-secure

Kind regards, Gonzalo

— Reply to this email directly, view it on GitHub https://github.com/tensorflow/privacy/issues/349, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB7LCU7GVRS2JMITWENY63TWHGBUJANCNFSM6AAAAAARZV5VUU . You are receiving this because you are subscribed to this thread.Message ID: @.***>