search

Baqend / Orestes-Bloomfilter

Library of different Bloom filters in Java with optional Redis-backing, counting and many hashing options.
Other
843 stars 245 forks source link

Support for large bit set #10

Closed shrinandj closed 10 years ago

shrinandj commented 10 years ago

The current implementation of optimalM(double n, double p) returns an int. This limits the largest bit set that can be used when creating a bloom filter.

E.g. I tried to create a bloomFilter with n = 100M but that crashed because the # of bits required to support that do not fit into an int.

DivineTraube commented 10 years ago

Hi thanks a lot, this will be fixed in our next release scheduled for next week.

DivineTraube commented 10 years ago

Oops, sorry I was mistaken - unfortunately neither the Java BitSet nor Redis do support bitsets larger than 2^32 bits: http://docs.oracle.com/javase/8/docs/api/java/util/BitSet.html http://redis.io/commands/setbit

So I guess we are bound to Bloom filters no larger than 512MB.