blevesearch / zapx

Zap file format compatible with a future version of Bleve
Apache License 2.0
11 stars 12 forks source link

MB-57871: Recover panic by GetCardinality #259

Closed moshaad7 closed 3 weeks ago

moshaad7 commented 1 month ago
invalid memory address or nil pointer dereference

goroutine 266940 [running]:
github.com/blevesearch/bleve/v2/index/scorch.(*IndexSnapshotTermFieldReader).Count.func1()
    /home/couchbase/jenkins/workspace/toy-unix-simple/godeps/src/github.com/blevesearch/bleve/index/scorch/snapshot_index_tfr.go:181 +0x85
panic({0xf47c00, 0x1e2e650})
    /home/couchbase/.cbdepscache/exploded/x86_64/go-1.18.7/go/src/runtime/panic.go:838 +0x207
github.com/RoaringBitmap/roaring.(*Bitmap).GetCardinality(...)
    /home/couchbase/.cbdepscache/gomodcache/pkg/mod/github.com/!roaring!bitmap/roaring@v0.9.4/roaring.go:667
github.com/blevesearch/zapx/v15.(*PostingsList).Count(0xc00f171680?)
    /home/couchbase/jenkins/workspace/toy-unix-simple/godeps/src/github.com/blevesearch/zapx/posting.go:240 +0xd4
github.com/blevesearch/bleve/v2/index/scorch.(*IndexSnapshotTermFieldReader).Count(0xc01c9a0d60?)
    /home/couchbase/jenkins/workspace/toy-unix-simple/godeps/src/github.com/blevesearch/bleve/index/scorch/snapshot_index_tfr.go:191 +0xe3
// GetCardinality returns the number of integers contained in the bitmap
func (rb *Bitmap) GetCardinality() uint64 {
    size := uint64(0)
    for _, c := range rb.highlowcontainer.containers {
        size += uint64(c.getCardinality())
    }
    return size
}
c.getCardinality panicked because c was nil

We couldn't determine the data and operations that lead our bitmap to this situation where once of the container was nil. Adding panic recover method to log bitmap data and stats in order to position us better in case this rare bug strikes again.

moshaad7 commented 3 weeks ago

Closing it as bitmap Stat collection will also fail in case of a presence of a nil container in it. My apologies for missing this detail

Adding relevant code sections here

// GetCardinality returns the number of integers contained in the bitmap
func (rb *Bitmap) GetCardinality() uint64 {
    size := uint64(0)
    for _, c := range rb.highlowcontainer.containers {
        size += uint64(c.getCardinality())
    }
    return size
}
// Stats returns details on container type usage in a Statistics struct.
func (rb *Bitmap) Stats() Statistics {
    stats := Statistics{}
    stats.Containers = uint64(len(rb.highlowcontainer.containers))
    for _, c := range rb.highlowcontainer.containers {
        stats.Cardinality += uint64(c.getCardinality())
        ...
    }
    return stats
}