PipelineDP is a Python framework for applying differentially private aggregations to large datasets using batch processing systems such as Apache Spark, Apache Beam, and more.
This PR implements estimation of RMSE from DatasetHistogram for l0_bound and linf_bound
The algorithm is the following
From l0_bound and l0_contributions_histogram the ratio data_dropped_from_l0 contribution bounding is computed.
From linf_bound and linf_contributions_histogram the ratio_data_dropped_from_linf contribution bounding is computed.
The total 'ratio_data_dropped' for contribution bounding is estimated from data_dropped_from_l0 and ratio_data_dropped_from_linf.
Then under the assumption that contribution bounding drops data uniformly on all partitions, for a partition of the size n, it is assumed that n*ratio_data_dropped data points are dropped with contribution bounding. And RMSE for this partition is computed as sqrt((n*ratio_data_dropped)**2 + noise_std**2)
This PR implements estimation of RMSE from
DatasetHistogramforl0_boundandlinf_boundThe algorithm is the following
l0_boundandl0_contributions_histogramthe ratiodata_dropped_from_l0contribution bounding is computed.linf_boundandlinf_contributions_histogramtheratio_data_dropped_from_linfcontribution bounding is computed.data_dropped_from_l0andratio_data_dropped_from_linf.n, it is assumed thatn*ratio_data_droppeddata points are dropped with contribution bounding. And RMSE for this partition is computed assqrt((n*ratio_data_dropped)**2 + noise_std**2)