figure out privacy accumulation

[ooovi]

we know the privacy guarantees for one training step, we need to infer that of the whole training procedure. we could:

• adjust the noise to the number of steps (if we are willing to force that to be constant instead of training until some accuracy is reached)
• use a privacy accountant as done here (and everywhere else i think)

[ooovi]

advanced composition gives us the bound for non stochastic GD: https://programming-dp.com/ch6.html#advanced-composition-for-approximate-differential-privacy

client side subsampling will allow us to get a lower bound like with the accountant, but we can't just use the accountant from the linked paper because in the non federated DP SGD samples once & computes the sample gradient & then clips and noises, while we sample multiple times & compute the sample gradients& sum them (locally), and then clip and noise (on the server)