search

world-federation-of-advertisers / cardinality_estimation_evaluation_framework

Evaluation framework and methods for estimating cardinalities of groups of sets
Apache License 2.0
22 stars 9 forks source link

Add configs to split privacy budget over multiple queries #109

Closed pasin30055 closed 3 years ago

pasin30055 commented 4 years ago

Add configs that allow experimenting with (hypothetical) multiple queries scenario, where we would like all the estimates to together be (epsilon, delta)-differentially private. For geometric noiser, this will just divide the budget over all queries. For gaussian, this results in a growth of sqrt(# of queries) in the amount of noise added.

An example run can be done via

python src/evaluations/run_evaluation.py \
--evaluation_out_dir="eval_results" \
--analysis_out_dir="eval_results" \
--report_out_dir="eval_results" \
--evaluation_config=smoke_test \
--num_runs=1 \
--evaluation_run_name="new_results" \
--sketch_estimator_configs=exp_bloom_filter-100000_10-first_moment_exp-no_local_dp-global_dp_1.0986,0.0000100-gaussian_noise-budget_split-1 \
--sketch_estimator_configs=exp_bloom_filter-100000_10-first_moment_exp-no_local_dp-global_dp_1.0986,0.0000100-geometric_noise-budget_split-1 \
--sketch_estimator_configs=exp_bloom_filter-100000_10-first_moment_exp-no_local_dp-global_dp_1.0986,0.0000100-gaussian_noise-budget_split-100 \
--sketch_estimator_configs=exp_bloom_filter-100000_10-first_moment_exp-no_local_dp-global_dp_1.0986,0.0000100-geometric_noise-budget_split-100
pasin30055 commented 4 years ago

Thanks Xichen for reviewing. I made some changes due to conflict with your previously merged pull request. Can you please take a look again and see if this looks ok? When you finish, can you also please help merge this PR? (I have no write access.)