Add a Cuckoo filter as an alternative to the Bloom filter - Githubissues

ukwa / ukwa-heritrix

The UKWA Heritrix3 custom modules and Docker builder.

9 stars 7 forks source link

Add a Cuckoo filter as an alternative to the Bloom filter #55

Open anjackson opened 4 years ago

anjackson commented 4 years ago

Cuckoo filters are similar to Bloom filters but allow entries to be removed. We’ll need an implementation that is

suitably licensed, Apache 2 to
thread-safe
reasonably fast
reasonably actively supported

Options

https://github.com/topics/cuckoo-filter?l=java
https://github.com/MGunlogson/CuckooFilter4J
https://github.com/cinnom/nano-cuckoo (relies on Unsafe.allocateMemory so not available on all platforms and may be deprecated in the near future)
https://github.com/bdupras/guava-probably (Google Guava 19.0 or higher, which might be an issue for the H3 codebase)