how the global privacy budget is calculated?

[AlphaPav]

Hi authors,

Thanks for releasing the code, which is very helpful. I have a question about the global privacy budget: Each local optimizer is attached to a separate PrivacyEngine, so we can track the privacy budget for each local client independently during the local SGD. https://github.com/ipc-lab/private-ml-for-health/blob/98b66920c659422bc963d70964f780b76fa67a48/private_training/src/update_s4.py#L90 The global optimizer is attached to another global PrivacyEngine. However, the global optimizer is not used because the server does not actually train the model and it only performs the aggregation. Given that the global optimizer is not used in practise, how the function "global_privacy_engine.get_privacy_spent(args.delta) " calculate an epsilon? https://github.com/ipc-lab/private-ml-for-health/blob/98b66920c659422bc963d70964f780b76fa67a48/private_training/src/federated_main_s4.py#L189

Thanks!

[mmalekzadeh]

Hey @AlphaPav

So, it is a sort of hacking we did to be able to use the Opacus library and simulate the proposed FL scenario.

If you notice this line: https://github.com/ipc-lab/private-ml-for-health/blob/98b66920c659422bc963d70964f780b76fa67a48/private_training/src/federated_main_s4.py#L111 Here for each local optimizer, we set the noise such that it simulates line 12 in Algorithm 1 in the paper. So, the main job of noise addition and other steps of DPSGD is done via PrivacyEngine of local optimizers.

Then, we create a global optimizer and we use that optimizer ONLY for computing privacy budget spent. If you notice this line: https://github.com/ipc-lab/private-ml-for-health/blob/98b66920c659422bc963d70964f780b76fa67a48/private_training/src/federated_main_s4.py#L134 The noise that we set for the global optimizer is set to the effective noise after aggregation. See Lemma 1 in the paper.

So, you are right. the global optimizer is not used because the server does not actually train the model and it only performs the aggregation. However, if you go deep into the Opacus library and see the method get_privacy_spent(), https://github.com/pytorch/opacus/blob/295a4806251a0295feb259612431b900d8efa73d/opacus/privacy_analysis.py#L270 you see that for computing the privacy budgets you do not need to actually train a model. What you need is to set the noise and other parameters correctly. This is the reason why we create a global PrivacyEngine and set its parameters according to our setting, and at each epoch, we only need to set the current steps, here https://github.com/ipc-lab/private-ml-for-health/blob/98b66920c659422bc963d70964f780b76fa67a48/private_training/src/federated_main_s4.py#L188 Then, as other parameters are already set, at each epoch we can compute the privacy budget.

Also, notice that in our experiment for this specific setting, every 30 epochs is equal to one actual epoch (By actual epoch I mean one round of visiting all samples in the training dataset). This is explained in this file: https://github.com/ipc-lab/private-ml-for-health/tree/98b66920c659422bc963d70964f780b76fa67a48/private_training Note that in FSCDP --epochs=30001 is actually the number of iterations and not epochs. Based on the setting, --epochs=30001 in FSCDP is similar to having 100 epochs in other FPDP and F setting

Hope this helps.

[mmalekzadeh]

Also, as a matter of fact, a pull request we made has just merged into the Opacus library which means, if you use the new version of Opacus, you will get a better privacy bound. Here is the PR https://github.com/pytorch/opacus/pull/162

[AlphaPav]

Hey @mmalekzadeh

Thanks for your timely and detailed response, which helps me tremendously. In addition, thanks for your solid paper, which addresses my question about "what is the privacy guarantee for the global model when each local agent performs DP-SGD".

Awesome. It's great to see you improve the Opacus library! :)

I have another question about Lemma 1. In order to achieve certain epsilon, Lemma 1 calculates the required local noise level. However, in the implementations, it seems that you can set arbitrary values for the noise level as a hyperparameter and calculate the epsilon dynamically at each epoch. Does the noise level in the experiments still satisfy the following equation from Lemma 1?

$ \sigma^{2}=\frac{2 \ln (1.25 / \delta) C^{2}}{\epsilon^{2}\left|\mathbb{D}_{k}^{t}\right|^{2} K}$

[mmalekzadeh]

@AlphaPav, happy to see that you find the paper and code helpful. Yes. You can set any amount of noise you want and achieve a corresponding epsilon. So, Lemma 1 is a general formulation and it is not bounded to a specific epsilon. You increase the noise and you get lower epsilon. I hope I answered your question.

[AlphaPav]

I see. Thank you!

[AlphaPav]

Hey @mmalekzadeh , I am not sure if the setting in FSCDP is correct: "Note that in FSCDP --epochs=30001 is actually the number of iterations and not epochs. Based on the setting, --epochs=30001 in FSCDP is similar to having 100 epochs in other FPDP and F setting"

In algorithm 1, each client performs one local SGD step. Lemma 1 gives the privacy cost for the global model after aggregation under one SGD step. Then we need to use composition for T SGD steps (T epochs in algorithm 1). So when you use moment accountant for the global model, the global privacy cost should be accumulated for each SGD step (each epoch), not per 30 epochs :

https://github.com/ipc-lab/private-ml-for-health/blob/98b66920c659422bc963d70964f780b76fa67a48/private_training/src/federated_main_s4.py#L179

https://github.com/ipc-lab/private-ml-for-health/blob/98b66920c659422bc963d70964f780b76fa67a48/private_training/src/federated_main_s4.py#L188

(In your FPDP, the moment accountant is used for the local model. The local privacy cost is also accumulated for each SGD step, not for each local epoch, which echos my opinion.)

Is my understanding correct? Do I miss something?

[mmalekzadeh]

You are right that the global privacy cost should be accumulated for each SGD step and this is exactly what we do in this line: https://github.com/ipc-lab/private-ml-for-health/blob/98b66920c659422bc963d70964f780b76fa67a48/private_training/src/federated_main_s4.py#L188

Notice that in this FSCDP setting, epoch is the iteration number (T in algorithm 1) and not the epoch as we used to. So, when we do global_privacy_engine.steps = epoch+1 we actually add current number of iterations. Sorry for bad naming :-)

[AlphaPav]

Yeah, that is clear and you are correct! Thanks for the reply.

Btw in FL with Parallel DP (FPDP), each client has its own eps_i, did you use the max eps_i among all clients as the global eps to plot Figure 1?

[mmalekzadeh]

Yes. Exactly. In parallel DP, the ultimate bound is decided via the maximum bound.

[AlphaPav]

Thank you!