About clip method.

[fourcake]

Hello, I noticed that in your implementation of "fed_avg_dp.py", the forward propagation on the client side is done by propagating samples one by one, calculating gradients, clipping, and then adding noise to the gradients of a batch. I would like to ask if it is possible to directly clip the gradients obtained by backpropagation of a batch with a fixed clipping threshold C and then add noise. What are the advantages and disadvantages of this method compared to your method? Hope to get your reply！

[JeffffffFu]

In sample level, you need to bound the sensitivity of each sample. you can not achieve through clipping the gradients of a batch.

------------------ 原始邮件 ------------------ 发件人: "JeffffffFu/Awesome-Differential-Privacy-and-Meachine-Learning" @.>; 发送时间: 2024年9月24日(星期二) 晚上7:34 @.>; @.***>; 主题: [JeffffffFu/Awesome-Differential-Privacy-and-Meachine-Learning] About clip method. (Issue #9)

Hello, I noticed that in your implementation of "fed_avg_dp.py", the forward propagation on the client side is done by propagating samples one by one, calculating gradients, clipping, and then adding noise to the gradients of a batch. I would like to ask if it is possible to directly clip the gradients obtained by backpropagation of a batch with a fixed clipping threshold C and then add noise. What are the advantages and disadvantages of this method compared to your method? Hope to get your reply！

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: @.***>

[fourcake]

Thank you for your answer. So in client-level DP, although the noise scale is calculated in a similar way, it is valid to directly clip and add noise to the gradient obtained from the client batch data, is that right?

[JeffffffFu]

I recommend you read this survey