ethereum-optimism / op-geth

GNU Lesser General Public License v3.0
255 stars 653 forks source link

Node stopped syncing after oom kill #255

Open alexqrid opened 4 months ago

alexqrid commented 4 months ago

System information

Geth version: v1.101304.2 Op-node version: v1.3.2 OS & Version: Linux

Expected behaviour

Node is syncing normally

Actual behaviour

Node stopped syncing and concerns about gap in the chain between ancients

Steps to reproduce the behaviour

Just run the node

Backtrace

I am running optimism stack in docker - https://github.com/smartcontracts/simple-optimism-node My node was syncing normally, but after oom kill due to high memory consumption it couldn't sync again, reporting that there is a problem with database, attached the logs below. I've upgraded op-node to v1.4.3 and op-geth to 1.101305.3, unfortunately it didn't help.

op-geth logs ```` errexit on noglob off ignoreeof off monitor off noexec off xtrace off verbose off noclobber off allexport off notify off nounset off errtrace off vi off pipefail off INFO [02-26|10:24:19.147] Enabling metrics collection INFO [02-26|10:24:19.149] Maximum peer count ETH=0 LES=0 total=0 INFO [02-26|10:24:19.150] Smartcard socket not found, disabling err="stat /run/pcscd/pcscd.comm: no such file or directory" INFO [02-26|10:24:19.153] Enabling recording of key preimages since archive mode is used WARN [02-26|10:24:19.153] Disabled transaction unindexing for archive node INFO [02-26|10:24:19.153] Set global gas cap cap=50,000,000 INFO [02-26|10:24:19.154] Initializing the KZG library backend=gokzg INFO [02-26|10:24:19.195] Allocated trie memory caches clean=307.00MiB dirty=0.00B INFO [02-26|10:24:21.419] Using leveldb as the backing database INFO [02-26|10:24:21.419] Allocated cache and file handles database=/geth/geth/chaindata cache=512.00MiB handles=524,288 INFO [02-26|10:24:36.001] Using LevelDB as the backing database INFO [02-26|10:24:36.001] Found legacy ancient chain path location=/geth/geth/chaindata/ancient INFO [02-26|10:24:36.046] Opened ancient database database=/geth/geth/chaindata/ancient readonly=false Chain metadata databaseVersion: 8 (0x8) headBlockHash: 0x284dc3a53bf254cabf3d22cfc3a8664fc5957db4dca4b433866ebb1612147c07 headFastBlockHash: 0x284dc3a53bf254cabf3d22cfc3a8664fc5957db4dca4b433866ebb1612147c07 headHeaderHash: 0x284dc3a53bf254cabf3d22cfc3a8664fc5957db4dca4b433866ebb1612147c07 lastPivotNumber: len(snapshotSyncStatus): 0 bytes snapshotDisabled: false snapshotJournal: 1891973 bytes snapshotRecoveryNumber: snapshotRoot: 0xf761dd8a26b2fefceffa3751f06614abafb9d256cfc9985f61b637eae126df29 txIndexTail: 0 (0x0) fastTxLookupLimit: Fatal: Failed to register the Ethereum service: gap in the chain between ancients [0 - #116579847] and leveldb [#116669875 - #116669874] Fatal: Failed to register the Ethereum service: gap in the chain between ancients [0 - #116579847] and leveldb [#116669875 - #116669874] ````