This PR addresses the worst-case behaviour that we have seen in the new micro-benchmark from #295. Also, this PR resolves #204.
With the fix, average search time and allocation for construction and search go up a bit, but the worst-case scenario that appsWithNearDups exercises drastically improves. search is back to allocating zero bytes, and allocations for construction are more than halved. Moreover, it is not clearly visible from the benchmark results, but the max RSS of the program goes down too.
Benchmark before after-after
Bench.Database.LSMTree.Internal.IndexCompact/construction/construction with 0-bit rfprec and chunk size 100 0.408e-4 0.427e-4 +4.69%
Bench.Database.LSMTree.Internal.IndexCompact/construction/construction with 16-bit rfprec and chunk size 100 1.080e-4 1.145e-4 +6.02%
Bench.Database.LSMTree.Internal.IndexCompact/construction/unsafeWriteRange-10k 0.580e-6 0.581e-6 +0.07%
Bench.Database.LSMTree.Internal.IndexCompact/construction/unsafeWriteRange-1k 0.937e-7 0.945e-7 +0.81%
Bench.Database.LSMTree.Internal.IndexCompact/non-uniformity/construct appsWithNearDups 2.098e-4 0.441e-4 -78.99%
Bench.Database.LSMTree.Internal.IndexCompact/non-uniformity/construct appsWithoutNearDups 2.002e-5 2.080e-5 +3.86%
Bench.Database.LSMTree.Internal.IndexCompact/non-uniformity/search appsWithNearDups 0.939e-3 0.042e-3 -95.55%
Bench.Database.LSMTree.Internal.IndexCompact/non-uniformity/search appsWithoutNearDups 2.492e-5 3.759e-5 +50.87%
Bench.Database.LSMTree.Internal.IndexCompact/searches/searches with 0-bit rfprec 0.386e-4 0.558e-4 +44.58%
Bench.Database.LSMTree.Internal.IndexCompact/searches/searches with 16-bit rfprec 0.980e-5 1.660e-5 +69.38%
Geometric mean 1.972e-5 1.430e-5 -27.50%
This PR addresses the worst-case behaviour that we have seen in the new micro-benchmark from #295. Also, this PR resolves #204.
With the fix, average search time and allocation for construction and search go up a bit, but the worst-case scenario that
appsWithNearDups
exercises drastically improves.search
is back to allocating zero bytes, and allocations for construction are more than halved. Moreover, it is not clearly visible from the benchmark results, but the max RSS of the program goes down too.RE allocations, compare with https://github.com/IntersectMBO/lsm-tree/pull/295#issue-2409218713