`Antiquary.IncrementBeaconState` panics inside `ef.Get`

[awskii]

INFO[07-22|10:57:31.931] P2P                                      app=caplin peers=71
INFO[07-22|10:57:35.793] Node is still syncing... downloading past blocks app=caplin stage=DownloadHistoricalBlocks slot=4369140 blockNumber=20299995 frozenBlocks=20299999 blk/sec=128292.7 snapshots=9499999
INFO[07-22|10:57:36.000] Ready to insert history, waiting for sync cycle to finish app=caplin stage=DownloadHistoricalBlocks
INFO[07-22|10:57:36.001] [Caplin] Forward Sync                    app=caplin stage=ForwardSync from=9568192 to=9568486
INFO[07-22|10:57:55.868] Full backfilling finished                app=caplin stage=DownloadHistoricalBlocks
INFO[07-22|10:57:55.906] Starting state processing                from=1 to=9568223
panic: runtime error: index out of range [3125] with length 3125

goroutine 113837 [running]:
github.com/erigontech/erigon-lib/recsplit/eliasfano32.(*EliasFano).get(0xc0c18c0a00, 0x0?)
        github.com/erigontech/erigon-lib@v0.0.0-00010101000000-000000000000/recsplit/eliasfano32/elias_fano.go:176 +0x36d
github.com/erigontech/erigon-lib/recsplit/eliasfano32.(*EliasFano).Get(...)
        github.com/erigontech/erigon-lib@v0.0.0-00010101000000-000000000000/recsplit/eliasfano32/elias_fano.go:187
github.com/erigontech/erigon-lib/recsplit.(*Index).OrdinalLookup(...)
        github.com/erigontech/erigon-lib@v0.0.0-00010101000000-000000000000/recsplit/index.go:357
github.com/erigontech/erigon/turbo/snapshotsync/freezeblocks.(*beaconSnapshotReader).ReadBlockBySlot(0xc17e677000, {0xc28a1736e0?, 0x29240?}, {0x323e738, 0xc28c5f7810}, 0x1869f)
        github.com/erigontech/erigon/turbo/snapshotsync/freezeblocks/beacon_block_reader.go:110 +0x6a5
github.com/erigontech/erigon/cl/antiquary.(*Antiquary).IncrementBeaconState(0xc0a7963a00, {0x3217660, 0xc19a5af810}, 0x91ffdf)
        github.com/erigontech/erigon/cl/antiquary/state_antiquary.go:268 +0xdf9
github.com/erigontech/erigon/cl/antiquary.(*Antiquary).loopStates(0xc0a7963a00, {0x3217660, 0xc19a5af810})
        github.com/erigontech/erigon/cl/antiquary/state_antiquary.go:87 +0x2d7
created by github.com/erigontech/erigon/cl/antiquary.(*Antiquary).Loop in goroutine 113763
        github.com/erigontech/erigon/cl/antiquary/antiquary.go:188 +0x103f

Erigon version af2a4a069f2389a3e9b968a1db3b50c046f189fd (main @ 19 Jul)

Removal of erigon-data/snapshots/*beaconblocks.idx and restart with caplin.backfilling parameter ended up with

[Gossip] Subscribed to topic             topic=/eth2/6a95a1a9/blob_sidecar_5/ssz_snappy
EROR[07-22|12:37:03.427] Antiquary failed                         app=caplin err="idx: index panic: at=v1-000000-000100-beaconblocks.seg, file: v1-000000-000100-beaconblocks.seg, runtime error: slice bounds out of range [:-1], [decompress.go:670 panic.go:770 panic.go:140 decompress.go:694 type.go:439 caplin_snapshots.go:57 caplin_snapshots.go:583 antiquary.go:117 run.go:275 asm_amd64.s:1695], [type.go:398 panic.go:770 d
ecompress.go:670 panic.go:770 panic.go:140 decompress.go:694 type.go:439 caplin_snapshots.go:57 caplin_snapshots.go:583 antiquary.go:117 run.go:275 asm_amd64.s:1695]"

So probably the issue is withing those caplin snapshot files which i raised a month ago.

[AskAlexSharov]

@awskii remove .idx of beacon files. it helped me. 2 files was recently re-released. https://github.com/erigontech/erigon-snapshot/pull/225

[AskAlexSharov]

fixed by removing some beaconblock .seg