Baqend / Orestes-Bloomfilter

Library of different Bloom filters in Java with optional Redis-backing, counting and many hashing options.
Other
843 stars 245 forks source link

Can't have multiple bloom filters and reconnect to existing bloom filters in Redis #13

Closed ankbul closed 10 years ago

ankbul commented 10 years ago

Hi,

I'm in a distributed environment where servers will be adding to shared bloom filters. Currently, there's no way to reference an existing bloom filter if a server disconnects and then wants to reconnect. It seems as if two instantiations will connect to the same bloom filter.

For example, consider this use-case:

CBloomFilterRedisBits<String> f1 = new CBloomFilterRedisBits<>(host, port, 100, 0.1, 4);
f1.add("dog");

CBloomFilterRedisBits<String> f2 = new CBloomFilterRedisBits<>(host, port, 100, 0.1, 4);
f2.add("cat");

f1.contains("dog"); // returns true, as expected
f1.contains("cat"); // also returns true, which is unexpected. It should not contain cat.

f2.contains("dog"); // also returns true, which is unexpected. It should not contain dog.
f2.contains("cat"); // return true, as expected

What is needed is an ability to associate a key to the bloom filter. That would allow us to reference the bloom filter after a disconnect.

ankbul commented 10 years ago

I've hacked a solution to this problem by implementing a key provider. You can view the changeset here:

https://github.com/scopely/Orestes-Bloomfilter/tree/redis-bits-multiple-filter-support

DivineTraube commented 10 years ago

Hi thanks for your code. We are adding this feature (to be released next week). It will introduce named Bloom filters as well as better concurrency and error handling.

DivineTraube commented 10 years ago

Hi, you can now have multiple Bloomfilters!