search

privacytrustlab / ml_privacy_meter

Privacy Meter: An open-source library to audit data privacy in statistical and machine learning algorithms.
MIT License
556 stars 99 forks source link

Improve the Privacy Meter Efficiency #89

Closed changhongyan123 closed 1 year ago

changhongyan123 commented 1 year ago

In this update, I updated the following aspects of the privacy meter.

  1. Fix the minor bugs of the privacy meter for the reference attack when using different hypothesis testing threshold functions.
  2. Improve the efficiency of the privacy meter: changing the for loop computation to the matrix computation. On the demo cases, the population attack's (population_metric.ipynb) running time reduces from 11 seconds with 500 fprs to 2 seconds. The reference attack's (reference_metric.ipynb) running time reduces from 123 seconds with 500 fprs to 5 seconds with 500 fprs. For the comparison notebook (attack_via_reference_or_distill.ipynb), the total time reduces from 2474 seconds to 2 seconds.
  3. Enable the gradient norm computation for pytorch model, based on the opacus GradSampleModule.
  4. Allow the user to determine whether to save the log file.
  5. Add the PytorchTensor model class to support the computation on the dataset used with Tensor.