Bloom filters can approach 600Kbytes. They are not used during compaction or iterator only work. A Get() request that opens a .sst file would likely process fewer bytes by actually reading the potential data block, than first unpacking the bloom filter. Memory tight machines might never open the bloom filter if jumping rapidly between .sst table files.
This branch reads the bloom filter only upon the second read of a .sst table file.
This branch also corrects a write throttle problem detected during testing of the bloom filter change.
paulplace
commented
7 years ago
Bloom filters can approach 600Kbytes. They are not used during compaction or iterator only work. A Get() request that opens a .sst file would likely process fewer bytes by actually reading the potential data block, than first unpacking the bloom filter. Memory tight machines might never open the bloom filter if jumping rapidly between .sst table files.
This branch reads the bloom filter only upon the second read of a .sst table file.
This branch also corrects a write throttle problem detected during testing of the bloom filter change.
Full details here:
https://github.com/basho/leveldb/wiki/mv-delayed-bloom