world-federation-of-advertisers / cardinality_estimation_evaluation_framework

Evaluation framework and methods for estimating cardinalities of groups of sets
Apache License 2.0
21 stars 9 forks source link

[Part 3]Fix bug: Fix Uniform BF out of bound error. #113

Closed huangxichen1 closed 3 years ago

huangxichen1 commented 3 years ago

This PR fix the uniform BF estimator error when the sketch is full or the sum of registers is negative.

With this fix, we can run through the interoperability test for the meta VoC estimator for the BF. The root cause of this error is that the test has a small universe size, and the estimated number of active registers could have huge variance under the local DP theme. As such, the fake uniform BF could be full and throw errors. In the smoke test and the complete eval, we didn't observe this because the universe size is 200 and 1000 times larger than that of the interoperability test.

IMPORTANT: Please merge the [Part X]Fix bug... PR in the reverse order, as they are dependent. Eg, merge [Part 3]Fix bug into [Part 2]Fix bug first, then [Part 2]Fix bug into [Part 1]Fix bug.