A suggestion of code improvement for BloomFilter.

longlong354 commented 1 month ago

API(s)

com.google.common.hash.BloomFilter::optimalNumOfBits(long n, double p)
&
com.google.common.hash.BloomFilter::optimalNumOfHashFunctions(long n, long m)

How do you want it to be improved?

1. use static caculated value of log(2) and squared log(2) :

        private static final double LOG_TWO = Math.log(2);
    private static final double SQUARED_LOG_TWO = Math.pow(LOG_TWO,2);

2. calculate optimalNumOfBits by static values:

    static long optimalNumOfBits(long n, double p) {
        if (p == 0) {
            p = Double.MIN_VALUE;
        }
        return (long) (-n * Math.log(p) / SQUARED_LOG_TWO);
    }

3. caculate optimalNumOfHashFunction by false positive rate(p) directly and using LOG_TWO :

        static int optimalNumOfHashFunctions(double p){
        return  Math.max(1, (int) Math.round( - Math.log(p) / LOG_TWO));
    }

Why do we need it to be improved?

Slightly reduce the method's execution time.
Refine the calculation of the optimalNumOfHashFunctions method to be more concise and efficient. The current code may lead people to mistakenly believe that the result is related to the number of samples(n) and the number of bits（m）, when in fact it is only related to the false positive rate(p).

Example

as the given codes in  “How do you want it to be improved”

Current Behavior

as the source codes in “How do you want it to be improved”

Desired Behavior

as the given codes in com.google.common.hash.BloomFilter::optimalNumOfBits(long n, double p) & com.google.common.hash.BloomFilter::optimalNumOfHashFunctions(long n, long m)

Concrete Use Cases

as "How do you want it to be improved"

Checklist

[X] I agree to follow the code of conduct.
[X] I have read and understood the contribution guidelines.
[X] I have read and understood Guava's philosophy, and I strongly believe that this proposal aligns with it.

eamonnmcmanus commented 1 month ago

This appears to have been a deliberate change back in 2012. Maybe @kluever remembers the rationale.

longlong354 commented 1 month ago

Thanks for reply. My fault didn't express the main point clearly enough. The main recommendation is using

optimalNumOfHashFunctions(double p)

instead of

optimalNumOfHashFunctions(long n, long m)

Ayush-Thakur-geek commented 2 weeks ago

Hey, @longlong354

So, you just want the usage of (double p) and not (long m, long n) as argument for bloomfilter.

eamonnmcmanus commented 2 weeks ago

The situation now is roughly that we start from $n$, the expected number of entries, and $p$, the desired false positive probability, and we derive $m$, the optimal number of bits, as

$$ m = \lfloor {-n\ \ln\ p } / { (\ln\ 2)^2} \rfloor $$

Then we further derive $k$, the optimal number of hash functions, as

$$ k = (m / n)\ \ln\ 2 \approx {-n\ \ln\ p } / { \ln\ 2} $$

rounded to an integer. You are proposing instead

$$ k = { -\ln\ p } / { \ln\ 2 } $$

removing the factor of $n$. That's a different number. Are you saying that it's more accurate? Could you explain why?

longlong354 commented 2 weeks ago

The derivation of the formula

k = (m / n) ln 2 ≈ -n ln p / ln 2

is inaccurate; it should be

k = (m / n) ln 2 ≈ - ln p / ln 2

Please kindly note that the bolded 'n' in the numerator cancels out with the 'n' in m( m = ⌊ − n ln ⁡ p / ( ln ⁡ 2 ) 2 ⌋ )"

ps: refering to https://en.wikipedia.org/wiki/Bloom_filter#Optimal_number_of_hash_functions also can be verified in https://krisives.github.io/bloom-calculator/ that n does not affect k

longlong354 commented 2 weeks ago

Hey, @longlong354

So, you just want the usage of (double p) and not (long m, long n) as argument for bloomfilter.

yep~

google / guava