search

FastFilter / fastfilter_java

Fast Approximate Membership Filters (Java)
Apache License 2.0
238 stars 27 forks source link

Attempt to build a filter with 50'000'000 elements fails #40

Closed catap closed 10 months ago

catap commented 11 months ago

I tried to build a filter with 50'000'000 elements with 48 bits per key. It fails on

COUNTING_BLOOM as:

java.lang.ArrayIndexOutOfBoundsException: Index 154880291 out of bounds for length 150000001
        at org.fastfilter.bloom.count.CountingBloom.add(CountingBloom.java:60)
        at org.fastfilter.bloom.count.CountingBloom.construct(CountingBloom.java:16)
        at org.fastfilter.FilterType$2.construct(FilterType.java:27)

SUCCINCT_COUNTING_BLOOM as:

java.lang.NegativeArraySizeException: -29608853
        at org.fastfilter.bloom.count.SuccinctCountingBloom$BitField.<init>(SuccinctCountingBloom.java:286)
        at org.fastfilter.bloom.count.SuccinctCountingBloom.<init>(SuccinctCountingBloom.java:74)
        at org.fastfilter.bloom.count.SuccinctCountingBloom.construct(SuccinctCountingBloom.java:26)
        at org.fastfilter.FilterType$3.construct(FilterType.java:33)

SUCCINCT_COUNTING_BLOOM_RANKED as:

java.lang.NegativeArraySizeException: -29608848
        at org.fastfilter.bloom.count.SuccinctCountingBloomRanked.<init>(SuccinctCountingBloomRanked.java:67)
        at org.fastfilter.bloom.count.SuccinctCountingBloomRanked.construct(SuccinctCountingBloomRanked.java:28)
        at org.fastfilter.FilterType$4.construct(FilterType.java:39)

BLOCKED_BLOOM as:

java.lang.NegativeArraySizeException: -29608847
        at org.fastfilter.bloom.BlockedBloom.<init>(BlockedBloom.java:37)
        at org.fastfilter.bloom.BlockedBloom.construct(BlockedBloom.java:16)
        at org.fastfilter.FilterType$5.construct(FilterType.java:45)

SUCCINCT_COUNTING_BLOCKED_BLOOM as:

java.lang.NegativeArraySizeException: -29608847
        at org.fastfilter.bloom.count.SuccinctCountingBlockedBloom.<init>(SuccinctCountingBlockedBloom.java:65)
        at org.fastfilter.bloom.count.SuccinctCountingBlockedBloom.construct(SuccinctCountingBlockedBloom.java:26)
        at org.fastfilter.FilterType$6.construct(FilterType.java:51)

SUCCINCT_COUNTING_BLOCKED_BLOOM_RANKED as:

java.lang.NegativeArraySizeException: -29608847
        at org.fastfilter.bloom.count.SuccinctCountingBlockedBloomRanked.<init>(SuccinctCountingBlockedBloomRanked.java:67)
        at org.fastfilter.bloom.count.SuccinctCountingBlockedBloomRanked.construct(SuccinctCountingBlockedBloomRanked.java:28)
        at org.fastfilter.FilterType$7.construct(FilterType.java:57)

GCS as:

java.lang.IllegalArgumentException
        at org.fastfilter.gcs.MonotoneList.generate(MonotoneList.java:48)
        at org.fastfilter.gcs.GolombCompressedSet.<init>(GolombCompressedSet.java:72)
        at org.fastfilter.gcs.GolombCompressedSet.construct(GolombCompressedSet.java:26)
        at org.fastfilter.FilterType$18.construct(FilterType.java:123)
lemire commented 11 months ago

Verified. I have a PR that should fix it.