Closed udi-speedb closed 1 year ago
Pull request: https://github.com/speedb-io/speedb/pull/163
Performance needs to cover a few tests. Checking with 20 BPK and compare to previous version, make sure no degradation. Test on 8 BPK and compare to native bloom.
Test on the 20 BPS show some degradation,
The second test was OK
The 8 BPS test
Pass performance tests
@erez says he has checked a branch named pared_bloom_filter-improved I will take a look at it now
During @noamhaham's performance runs, he discovered that there is a performance degradation with the paired filter with bpk=10. suggests to adjust the user's bpk (round up) to end up in an effective bpk that is larger that the user's. This adjustment implies more memory consumption for the filters (relative to not rounding up). The more keys a filter has, the smaller the adjustment / memory-consumption-increase and vice versa => Noam expects this to be felt more in Level 0 and less as the level progresses. However, since most of the data is in the lower levels, he expects the total consumption increase to be negligible.