error when execute privacy_accounting_fed_instancelevel.py

[Victor-QTP]

Hi! I am trying to reproduce the result for CIFAR10 in your paper [1]

1. When I ran the privacy_accounting_fed_instancelevel.py, I encontered some issues: Exception has occurred: ImportError cannot import name 'compute_heterogenous_rdp' from 'tensorflow_privacy.privacy.analysis.rdp_accountant' (/home/me/miniconda3/envs/dpsgd/lib/python3.8/site-packages/tensorflow_privacy/privacy/analysis/rdp_accountant.py) File "/media/me/me/PPFL/Fed-alphaCDP/DP_code/privacy_accounting_fed_instancelevel.py", line 212, in <module> from tensorflow_privacy.privacy.analysis.rdp_accountant import compute_heterogenous_rdp,get_privacy_spent

Steps to reproduce the error:

1. create the env with environment.yml.
2. run create_FLdistribution.sh
3. run privacy_accounting_fed_instancelevel.py => error.

It seems like the tensorflow privacy api's rdp_accountant you provided has a missing module? (compute_heterogenous_rdp) Could you please upload it?

Additionally, I would like to have 2 more questions regarding the Fed-CDP [1] implementation in Fed-CDP.py.

1. In the paper, C (l2_norm_clip) and σ (noise_multiplier/scale) were chosen to be 4 and 6, respectively. However, in the code (Fed-CDP.py), the σ's value was chosen to be 0.5, why it is so? This σ of 0.5 is used in the calculation of per model layer Gaussian Noises

            GaussianNoises = [
                1.0 / x.shape[0] * np.random.normal(loc=0.0, scale=float(noise_multiplier * l2_norm_clip),
                                                    size=MeanClippedgradients[i].shape) for i in
                range(num_weights)]  # layerwise gaussian noise

However,

In the last part of the code:

    with open('sigma.csv', mode='a') as sigma_file:
        writer_sigma = csv.writer(sigma_file, delimiter=',', quotechar='"', quoting=csv.QUOTE_MINIMAL)
        # writer_sigma.writerow([epoch, step, 24/sqrt(sum(n*n for n in l2_norm_clip)/len(l2_norm_clip))])  #if per exampl
        writer_sigma.writerow([round, local_iter, 24.0 / l2_norm_clip])  # if batch

2. It seems like the σ is calculated as 24.0/l2_norm_clip; what is this σ (is it the noise multiplier/scale?), and why it is equal to 24.0/l2_norm_clip (24.0/4=6) instead of 0.5?

Btw, I find your work to be fascinating, so could you please show me the instructions on how to reproduce the CIFAR-10 results in table 6 and 7?

Ref: [1] W. Wei, L. Liu, J. Zhou, K.-H. Chow, and Y. Wu, “Securing Distributed SGD against Gradient Leakage Threats.” arXiv, May 10, 2023. Accessed: Sep. 14, 2023. [Online]. Available: http://arxiv.org/abs/2305.06473

[WenqiWei789]

Hi, thank you for your interests of our work.

For your Q1 and Q2, this is because we are measuring difference choices of sigma and one choice of sigma is 0.5.

For Table 6, you will fix the noise scale sigma and only allow the dynamic sensitivity. Since the privacy spending is only related to the noise scale according to the theoretical analysis. Then the dynamic sensitivity will lead to better accuracy performance.

For Table 7, we still have the fixed noise scale and dynamic sensitivity. Given a fixed accuracy goal, it takes much quicker rounds to reach the accuracy goal than the fixed privacy parameters. In this way, the privacy spending is smaller, indicating better differential privacy guarantee.

For the module compute_heterogenous_rdp, the official repo of moments accountant has been depreciated. We will try our best to find the original file. The basic idea is to replace the fixed noise scale in the original rdp code with a list of sigma choices.

BTW, please use this reference instead: W. Wei, L. Liu, J. Zhou, K. -H. Chow and Y. Wu, "Securing Distributed SGD Against Gradient Leakage Threats," in IEEE Transactions on Parallel and Distributed Systems, vol. 34, no. 7, pp. 2040-2054, July 2023, doi: 10.1109/TPDS.2023.3273490.