PipelineDP is a Python framework for applying differentially private aggregations to large datasets using batch processing systems such as Apache Spark, Apache Beam, and more.
This PR implements estimation of RMSE from DatasetHistogram for l0_bound and linf_bound
The algorithm is the following
From l0_bound and l0_contributions_histogram the ratio data_dropped_from_l0 contribution bounding is computed.
From linf_bound and linf_contributions_histogram the ratio_data_dropped_from_linf contribution bounding is computed.
The total 'ratio_data_dropped' for contribution bounding is estimated from data_dropped_from_l0 and ratio_data_dropped_from_linf.
Then under the assumption that contribution bounding drops data uniformly on all partitions, for a partition of the size n, it is assumed that n*ratio_data_dropped data points are dropped with contribution bounding. And RMSE for this partition is computed as sqrt((n*ratio_data_dropped)**2 + noise_std**2)
This PR implements estimation of RMSE from
DatasetHistogram
forl0_bound
andlinf_bound
The algorithm is the following
l0_bound
andl0_contributions_histogram
the ratiodata_dropped_from_l0
contribution bounding is computed.linf_bound
andlinf_contributions_histogram
theratio_data_dropped_from_linf
contribution bounding is computed.data_dropped_from_l0
andratio_data_dropped_from_linf
.n
, it is assumed thatn*ratio_data_dropped
data points are dropped with contribution bounding. And RMSE for this partition is computed assqrt((n*ratio_data_dropped)**2 + noise_std**2)