search

treeverse / lakeFS

lakeFS - Data version control for your data lake | Git for data
https://docs.lakefs.io
Apache License 2.0
4.46k stars 360 forks source link

Crash in `PartitionIterator.Close()` #4108

Closed itaidavid closed 2 years ago

itaidavid commented 2 years ago

While itr field is checked for nil value but as it is an interface its type should be checked too Need to reproduce this as part of #3636 

2022/09/04 22:27:12 http: panic serving 10.0.41.133:32934: runtime error: invalid memory address or nil pointer dereference
goroutine 99962570 [running]:
net/http.(*conn).serve.func1()
    /usr/local/go/src/net/http/server.go:1802 +0xb9
panic({0x17af000, 0x34bf510})
    /usr/local/go/src/runtime/panic.go:1047 +0x266
github.com/treeverse/lakefs/pkg/kv/postgres.(*EntriesIterator).Close(0x7f51089b6108)
    /build/pkg/kv/postgres/store.go:352 +0x12
github.com/treeverse/lakefs/pkg/kv.(*PartitionIterator).Close(...)
    /build/pkg/kv/iterators.go:322
github.com/treeverse/lakefs/pkg/graveler/staging.(*Iterator).Close(0xc01a5b46b0)
    /build/pkg/graveler/staging/iterator.go:73 +0x2b
github.com/treeverse/lakefs/pkg/graveler.(*CombinedIterator).Close(0xc01b53eed0)
    /build/pkg/graveler/combined_iterator.go:130 +0x29
github.com/treeverse/lakefs/pkg/graveler.(*FilterTombstoneIterator).Close(0xc00e436a00)
    /build/pkg/graveler/combined_iterator.go:165 +0x22
github.com/treeverse/lakefs/pkg/catalog.(*valueEntryIterator).Close(0x143bf71)
    /build/pkg/catalog/value_entry_iterator.go:59 +0x22
github.com/treeverse/lakefs/pkg/catalog.(*prefixIterator).Close(0x143eb26)
    /build/pkg/catalog/prefix_iterator.go:58 +0x22
github.com/treeverse/lakefs/pkg/catalog.(*entryListingIterator).Close(0x143c07b)
    /build/pkg/catalog/entry_listing_iterator.go:53 +0x22
github.com/treeverse/lakefs/pkg/catalog.(*Catalog).ListEntries(0xc000113140, {0x28a5030, 0xc01c308cf0}, {0xc017198dd9, 0x11}, {0xc017198df0, 0x28}, {0xc01c6415f0, 0x85}, {0xc017198e38, ...}, ...)
    /build/pkg/catalog/catalog.go:809 +0x8ae
github.com/treeverse/lakefs/pkg/api.(*Controller).ListObjects(0xc00024c2d0, {0x28914e0, 0xc019629068}, 0xc01cbaf500, {0xc017198dd9, 0x11}, {0xc017198df0, 0x28}, {0xc0137cdbac, 0xc016a674c0, ...})
    /build/pkg/api/controller.go:2822 +0x246
github.com/treeverse/lakefs/pkg/api.(*ServerInterfaceWrapper).ListObjects.func1({0x28914e0, 0xc019629068}, 0x19f0001)
    /build/pkg/api/lakefs.gen.go:19333 +0x78
github.com/treeverse/lakefs/pkg/api.(*ServerInterfaceWrapper).ListObjects(0xc005f667e0, {0x28914e0, 0xc019629068}, 0xc01cbaf400)
    /build/pkg/api/lakefs.gen.go:19340 +0xb98
nopcoder commented 2 years ago

Crash caused because of interface compare with nil. The nil value is set by the a call to SeekGE that failed with an error.