We found big allocations while experimenting with block property filters at our workplace. My intuition is that such allocation is made while converting string to byte.
byteProps := []byte(props)
Here is the collected profile for better understanding.
ROUTINE ======================== github.com/cockroachdb/pebble/sstable.(*BlockPropertiesFilterer).IntersectsUserPropsAndFinishInit in /Users/marcin/go/pkg/mod/github.com/cockroachdb/pebble@v0.0.0-20221109022758-7b30bd86ff65/sstable/block_property.go
1.69GB 1.69GB (flat, cum) 14.71% of Total
. . 620: if !ok {
. . 621: // Collector was not used when writing this file, so it is
. . 622: // considered intersecting.
. . 623: continue
. . 624: }
1.68GB 1.68GB 625: byteProps := []byte(props)
. . 626: if len(byteProps) < 1 {
. . 627: return false, base.CorruptionErrorf(
. . 628: "block properties for %s is corrupted", f.filters[i].Name())
. . 629: }
. . 630: shortID := shortID(byteProps[0])
. . 631: intersects, err := f.filters[i].Intersects(byteProps[1:])
. . 632: if err != nil || !intersects {
. . 633: return false, err
. . 634: }
. . 635: // Intersects the sstable, so need to use this filter when
. . 636: // deciding whether to read blocks.
. . 637: n := len(f.shortIDToFiltersIndex)
. . 638: if n <= int(shortID) {
. . 639: if cap(f.shortIDToFiltersIndex) <= int(shortID) {
9.50MB 9.50MB 640: index := make([]int, shortID+1, 2*(shortID+1))
. . 641: copy(index, f.shortIDToFiltersIndex)
. . 642: f.shortIDToFiltersIndex = index
. . 643: } else {
. . 644: f.shortIDToFiltersIndex = f.shortIDToFiltersIndex[:shortID+1]
. . 645: }
Hi Team,
We found big allocations while experimenting with block property filters at our workplace. My intuition is that such allocation is made while converting string to byte.
Here is the collected profile for better understanding.
Jira issue: PEBBLE-167