search

XinFinOrg / XDPoSChain

Customer centric, Hybrid & Interoperable XinFin Network
https://www.xinfin.org
GNU Lesser General Public License v3.0
51 stars 61 forks source link

index allocation went beyond available resultCache space #217

Closed cryptarchnoble closed 6 months ago

cryptarchnoble commented 1 year ago

Encountered the following error in logs after syncing from bootstrap file on 12/22/2022:


INFO [12-22|20:57:52] Regenerated local transaction journal    transactions=0 accounts=0
INFO [12-22|20:57:52] Starting P2P networking 
INFO [12-22|20:57:55] UDP listener up                          self=enode://f57f0ebe03c836099593f1d1e99ff4c9305e9e9143d587e1d6dd95194f26d799b747e24227fe0ca59b23f6fd719f87a7a7227036a87fb8df0d478244d9916709@[::]:30303
INFO [12-22|20:57:55] Stats daemon started 
INFO [12-22|20:57:55] HTTP endpoint opened                     url=http://0.0.0.0:8545 cors=* vhosts=*
INFO [12-22|20:57:55] WebSocket endpoint opened                url=ws://[::]:8546
INFO [12-22|20:57:55] RLPx listener up                         self=enode://f57f0ebe03c836099593f1d1e99ff4c9305e9e9143d587e1d6dd95194f26d799b747e24227fe0ca59b23f6fd719f87a7a7227036a87fb8df0d478244d9916709@[::]:30303
INFO [12-22|20:57:55] IPC endpoint opened                      url=/work/xdcchain/XDC.ipc
INFO [12-22|20:57:56] Unlocked account                         address=xdce2bb8C66Eb33b40969E4Cec628c7638C104d66a8
INFO [12-22|20:57:56] Etherbase automatically configured       address=xdce2bb8C66Eb33b40969E4Cec628c7638C104d66a8
INFO [12-22|20:58:35] Block synchronisation started 
INFO [12-22|20:58:37] Persisted trie from memory database      nodes=0 size=0.00B time=5.972µs gcnodes=0 gcsize=0.00B gctime=0s livenodes=1 livesize=0.00B
INFO [12-22|20:58:37] Persisted trie from memory database      nodes=0 size=0.00B time=3.01µs  gcnodes=0 gcsize=0.00B gctime=0s livenodes=1 livesize=0.00B
INFO [12-22|20:58:37] Persisted trie from memory database      nodes=0 size=0.00B time=3.45µs  gcnodes=0 gcsize=0.00B gctime=0s livenodes=1 livesize=0.00B
INFO [12-22|20:58:37] Imported new chain segment               blocks=2 txs=6 mgas=0.000 elapsed=265.222ms mgasps=0.000 number=52978546 hash=873335…51e7b1 cache=11.46kB
INFO [12-22|20:58:38] Imported new chain segment               blocks=3 txs=104 mgas=0.000 elapsed=103.678ms mgasps=0.000 number=52978549 hash=af3c51…7b41db cache=226.65kB
INFO [12-22|20:58:38] Imported new chain segment               blocks=7 txs=7   mgas=0.000 elapsed=14.045ms  mgasps=0.000 number=52978556 hash=1a7a03…28a269 cache=226.65kB
INFO [12-22|20:58:40] Imported new chain segment               blocks=14 txs=121 mgas=0.073 elapsed=2.166s    mgasps=0.033 number=52978570 hash=1e85fb…db02a1 cache=476.13kB
You've encountered a sought after, hard to reproduce bug. Please report this to the developers <3 https://github.com/XinFinOrg/XDPoSChain/issues
index allocation went beyond available resultCache space
/work/xdcchain/eth/downloader/queue.go:510
goroutine 5112 [running]:
runtime/debug.Stack(0x2b, 0x0, 0x0)
        /usr/local/go/src/runtime/debug/stack.go:24 +0x9d
runtime/debug.PrintStack()
        /usr/local/go/src/runtime/debug/stack.go:16 +0x22
github.com/XinFinOrg/XDPoSChain/common.Report(0xc000a6ba28, 0x1, 0x1)
        /work/xdcchain/common/debug.go:35 +0x1a5
github.com/XinFinOrg/XDPoSChain/eth/downloader.(*queue).reserveHeaders(0xc0016d9300, 0xc032c4de00, 0x80, 0xc0142f0900, 0xc0000a0fa0, 0xc0142f0930, 0xc0142f0960, 0x148baa0, 0x49b1d8, 0xc009361230, ...)
        /work/xdcchain/eth/downloader/queue.go:510 +0x6cb
github.com/XinFinOrg/XDPoSChain/eth/downloader.(*queue).ReserveBodies(0xc0016d9300, 0xc032c4de00, 0x80, 0x0, 0xc032c4de00, 0x0, 0x0)
        /work/xdcchain/eth/downloader/queue.go:462 +0xeb
github.com/XinFinOrg/XDPoSChain/eth/downloader.(*Downloader).fetchParts(0xc005fe6300, 0x16acfc0, 0xc0000a5430, 0xc000081740, 0xc000a6be90, 0xc0002727e0, 0xc000a6be80, 0xc000a6bef0, 0xc000a6bee0, 0xc000a6bed0, ...)
        /work/xdcchain/eth/downloader/downloader.go:1116 +0x866
github.com/XinFinOrg/XDPoSChain/eth/downloader.(*Downloader).fetchBodies(0xc005fe6300, 0x3286371, 0x0, 0x0)
        /work/xdcchain/eth/downloader/downloader.go:937 +0x307
github.com/XinFinOrg/XDPoSChain/eth/downloader.(*Downloader).syncWithPeer.func4(0xf0de4ca587ae662, 0xa7bc248964728e32)
        /work/xdcchain/eth/downloader/downloader.go:466 +0x36
github.com/XinFinOrg/XDPoSChain/eth/downloader.(*Downloader).spawnSync.func1(0xc0334033d0, 0xc032dc5680, 0xc03345dde0)
        /work/xdcchain/eth/downloader/downloader.go:486 +0x53
created by github.com/XinFinOrg/XDPoSChain/eth/downloader.(*Downloader).spawnSync
        /work/xdcchain/eth/downloader/downloader.go:486 +0xc3
#### BUG! PLEASE REPORT ####
INFO [12-22|20:58:59] Imported new chain segment               blocks=3  txs=5   mgas=0.773 elapsed=18.213s   mgasps=0.042 number=52978573 hash=1466de…a830ef cache=512.71kB
INFO [12-22|20:59:06] Persisted trie from memory database      nodes=0 size=0.00B time=205.22µs gcnodes=4 gcsize=2.25kB gctime=16.765µs livenodes=4823 livesize=1.80mB
INFO [12-22|20:59:06] Persisted trie from memory database      nodes=0 size=0.00B time=1.825µs  gcnodes=0 gcsize=0.00B  gctime=412ns    livenodes=1    livesize=0.00B
INFO [12-22|20:59:06] Persisted trie from memory database      nodes=0 size=0.00B time=1.253µs  gcnodes=0 gcsize=0.00B  gctime=273ns    livenodes=1    livesize=0.00B
INFO [12-22|20:59:07] Imported new chain segment               blocks=104 txs=855 mgas=4.186 elapsed=8.075s    mgasps=0.518 number=52978677 hash=4809ea…99f613 cache=2.29mB
INFO [12-22|20:59:15] Imported new chain segment               blocks=115 txs=973 mgas=4.171 elapsed=8.207s    mgasps=0.508 number=52978792 hash=0fb15b…1451c1 cache=2.39mB
INFO [12-22|20:59:17] Persisted trie from memory database      nodes=723 size=265.67kB time=7.59196ms gcnodes=4118 gcsize=1.54mB gctime=9.662157ms livenodes=4951 livesize=1.83mB
INFO [12-22|20:59:17] Persisted trie from memory database      nodes=0   size=0.00B    time=4.973µs   gcnodes=0    gcsize=0.00B  gctime=24.724µs   livenodes=1    livesize=0.00B
INFO [12-22|20:59:17] Persisted trie from memory database      nodes=0   size=0.00B    time=2.693µs   gcnodes=0    gcsize=0.00B  gctime=20.137µs   livenodes=1    livesize=0.00B
INFO [12-22|20:59:22] Persisted trie from memory database      nodes=780 size=275.64kB time=6.244924ms gcnodes=4170 gcsize=1.56mB gctime=9.556615ms livenodes=4501 livesize=1.66mB
INFO [12-22|20:59:22] Persisted trie from memory database      nodes=0   size=0.00B    time=4.212µs    gcnodes=0    gcsize=0.00B  gctime=24.76µs    livenodes=1    livesize=0.00B
INFO [12-22|20:59:22] Persisted trie from memory database      nodes=0   size=0.00B    time=2.774µs    gcnodes=0    gcsize=0.00B  gctime=19.975µs   livenodes=1    livesize=0.00B
INFO [12-22|20:59:23] It's time to update new set of masternodes for the next epoch...```

This did not result in any problems. Node began syncing from other nodes after ~5 minutes.
gzliudan commented 1 year ago

I encounter this error 598 times when I sync from block 1 to block 35115300. The error is same as reported in #156. I'm using v1.4.6. My log messages are below:

INFO [06-07|16:44:29] Imported new chain segment               blocks=99  txs=884  mgas=0.616  elapsed=882.845ms mgasps=0.698  number=35114949 hash=195e0e…cd2e83 cache=0.00B
You've encountered a sought after, hard to reproduce bug. Please report this to the developers <3 https://github.com/XinFinOrg/XDPoSChain/issues
index allocation went beyond available resultCache space
/home/liudan/XDPoSChain/eth/downloader/queue.go:510
goroutine 7259132 [running]:
runtime/debug.Stack()
        /home/liudan/go-1.20.4/src/runtime/debug/stack.go:24 +0x65
runtime/debug.PrintStack()
        /home/liudan/go-1.20.4/src/runtime/debug/stack.go:16 +0x19
github.com/XinFinOrg/XDPoSChain/common.Report({0xc01523fa88, 0x1, 0x1})
        /home/liudan/XDPoSChain/common/debug.go:35 +0x12a
github.com/XinFinOrg/XDPoSChain/eth/downloader.(*queue).reserveHeaders(0xc00a3be000, 0xc00cceed20, 0x80, 0xc038e36270?, 0xc007df2028, 0x0?, 0xc01523faf0?, 0x133e628)
        /home/liudan/XDPoSChain/eth/downloader/queue.go:510 +0x5a5
github.com/XinFinOrg/XDPoSChain/eth/downloader.(*queue).ReserveBodies(0xcbac0c?, 0xcd34c6?, 0xc01523fb48?)
        /home/liudan/XDPoSChain/eth/downloader/queue.go:462 +0xf2
github.com/XinFinOrg/XDPoSChain/eth/downloader.(*Downloader).fetchParts(0xc00015e000, {0x14b7020, 0xc00025c060}, 0xc0000a3860, 0xc01523fea8, 0xc00a39c0e0, 0xc01523fe98, 0xc01523ff08, 0xc01523fef8, 0xc01523fee8, ...)
        /home/liudan/XDPoSChain/eth/downloader/downloader.go:1116 +0xd36
github.com/XinFinOrg/XDPoSChain/eth/downloader.(*Downloader).fetchBodies(0xc00015e000, 0x10d2820?)
        /home/liudan/XDPoSChain/eth/downloader/downloader.go:937 +0x2a8
github.com/XinFinOrg/XDPoSChain/eth/downloader.(*Downloader).syncWithPeer.func4()
        /home/liudan/XDPoSChain/eth/downloader/downloader.go:466 +0x25
github.com/XinFinOrg/XDPoSChain/eth/downloader.(*Downloader).spawnSync.func1()
        /home/liudan/XDPoSChain/eth/downloader/downloader.go:486 +0x68
created by github.com/XinFinOrg/XDPoSChain/eth/downloader.(*Downloader).spawnSync
        /home/liudan/XDPoSChain/eth/downloader/downloader.go:486 +0x6f
#### BUG! PLEASE REPORT ####
INFO [06-07|16:44:32] Imported new chain segment               blocks=350 txs=2850 mgas=7.884  elapsed=2.965s    mgasps=2.659  number=35115299 hash=6e1873…710b14 cache=0.00B
wjrjerome commented 7 months ago

hi @gzliudan does the change you made on your own branch resolve the issue? https://github.com/gzliudan/XDPoSChain/commit/0eba44363911ab78b3d660367fc8e94bafaa46fb

gzliudan commented 7 months ago

This is not a bug. Maybe we should change log messages.