divviup / janus

Experimental implementation of the Distributed Aggregation Protocol (DAP) specification.
Mozilla Public License 2.0
54 stars 15 forks source link

Strengthen integration tests of DP with statistical test on noise distribution #3325

Open divergentdave opened 4 months ago

divergentdave commented 4 months ago

Currently, the integration tests exercising DP just confirm that any noise was added to the aggregate result. We could strengthen this by performing a statistical test to evaluate whether the observed noise is a likely output of sampling the desired distribution, and failing if they are too different. As an example, libprio-rs currently uses a chi-squared test on a histogram produced from the discrete Gaussian sampler. (there may be other appropriate tests as well)

divergentdave commented 4 months ago

I looked at the distribution of $Lap\mathbb{Z}(t)+Lap\mathbb{Z}(t)$ a couple ways, and I wind up with either an infinite sum or an integral that seems hard to put in closed form. It should be possible to use numerical approximations to the PDF or CDF instead, and that'll be good enough for test purposes.