cockroachdb / pebble

RocksDB/LevelDB inspired key-value database in Go

BSD 3-Clause "New" or "Revised" License

4.6k stars 424 forks source link

TestMetaCrossVersion and TestMeta with no invariants failure #3678

Closed RaduBerinde closed 5 days ago

RaduBerinde commented 1 week ago

https://github.com/cockroachdb/pebble/actions/runs/9601942213/job/26481618161?pr=3677

        -iter20.SeekLT("dlrsqhqtkugm@10", "") // [true,"cearmwwagb@15","ojtyravuafvuep",["cearmwwagb","gsohc")=>{"@13"="jkzofmypok"}*] <nil> #1700
        +iter20.SeekLT("dlrsqhqtkugm@10", "") // [true,"cearmwwagb@3","dwyydffdbimw",["cearmwwagb","gsohc")=>{"@13"="jkzofmypok"}*] <nil> #1700

SEED=1718905245421630211 FACTOR=10 ./scripts/run-crossversion-meta.sh crl-release-24.1 HEAD

RaduBerinde commented 1 week ago

Couldn't reproduce with the command above :/

RaduBerinde commented 1 week ago

Possibly introduced by https://github.com/cockroachdb/pebble/pull/3672

RaduBerinde commented 1 week ago

I couldn't reproduce myself, but I found some cases of using FragmentIterator spans after Close which may be interacting with #3672

RaduBerinde commented 1 week ago

This was very tricky to debug. The finding here is that there is code that compares fragment iterators by pointer to see if there was a change. This doesn't work with a pool. https://github.com/cockroachdb/pebble/blob/master/merging_iter.go#L544

RaduBerinde commented 5 days ago

3689 fixed the main underlying issue. #3697 will close.