Stuck at 19800k block with sentry warnings

kovalishinilya commented 4 months ago

System information

Erigon version: 2.60.4-72ab70be

OS & Version: Ubuntu 24.04 LTS

Commit hash: 72ab70b

Erigon Command (with flags/config): erigon --datadir --db.size.limit 4TB --chain mainnet Consensus Layer:

Consensus Layer Command (with flags/config):

Chain/Network: Mainnet

Expected behaviour

sync normally

Actual behaviour

no syncing whatsoever stuck at block 19799999 experiencing sentry warnings

WARN[07-10|23:10:22.358] sentry.StatusDataProvider: The canonical chain current header not found in the database. Check the database consistency. Using genesis as a fallback. 
WARN[07-10|23:10:22.358] sentry.StatusDataProvider: The canonical chain current header not found in the database. Check the database consistency. Using genesis as a fallback. 
WARN[07-10|23:10:22.358] sentry.StatusDataProvider: The canonical chain current header not found in the database. Check the database consistency. Using genesis as a fallback. 
WARN[07-10|23:10:22.358] sentry.StatusDataProvider: The canonical chain current header not found in the database. Check the database consistency. Using genesis as a fallback. 
WARN[07-10|23:10:22.358] sentry.StatusDataProvider: The canonical chain current header not found in the database. Check the database consistency. Using genesis as a fallback. 
WARN[07-10|23:10:22.358] sentry.StatusDataProvider: The canonical chain current header not found in the database. Check the database consistency. Using genesis as a fallback. 
WARN[07-10|23:10:22.358] sentry.StatusDataProvider: The canonical chain current header not found in the database. Check the database consistency. Using genesis as a fallback. 
WARN[07-10|23:10:22.358] sentry.StatusDataProvider: The canonical chain current header not found in the database. Check the database consistency. Using genesis as a fallback.

after these two lines there are just memory and txpool stats

[snapshots:download] Stat                blocks=19800k indices=19800k alloc=3.1GB sys=4.8GB
INFO[07-10|23:10:25.230] [txpool] Started 
INFO[07-10|23:13:22.342] [p2p] GoodPeers                          eth68=2 eth66=1 eth67=3

Steps to reproduce the behaviour

restart erigon

Backtrace

kovalishinilya commented 4 months ago

also print_stages show that db has only blocks from 19800k although otterscan shows that last block on my node is 19799999

                 stage_at    prune_at
Snapshots            19799999    0
Headers              19799999    0
BorHeimdall              0       0
BlockHashes              19799999    0
Bodies               19799999    0
Senders              19799999    0
Execution            19799999    19799999
Translation              0       0
HashState            19799999    0
IntermediateHashes       19799999    19799999
AccountHistoryIndex          19799999    0
StorageHistoryIndex          19799999    0
LogIndex             19799999    0
CallTraces           19799999    19799999
TxLookup             19799999    19799999
Finish               19799999    0
--
prune distance: 

blocks.v2: true, blocks=19799999, segments=19799999, indices=19799999
blocks.bor.v2: segments=0, indices=0

history.v3: false,  idx steps: 0.00, lastBlockInSnap=0, TxNums_Index(0,1)

sequence: EthTx=2477320040, NonCanonicalTx=0

in db: first header 19800000, last header 20285456, first body 19800000, last body 20285456
--

AskAlexSharov commented 4 months ago

--internalcl

RickyEsclapon commented 4 months ago

Could someone explain this better? First time trying to run a node and this gave me massive difficulties. Did we do something wrong or what was the cause of this? I was also having the same exact problem on block 19799999 before eventually getting it synced with --internalcl

AskAlexSharov commented 4 months ago

All Ethereum clients have same architecture: 1 service for Consensus Layer, 1 service for Execution Layer. It's how The Merge was implemented/released: https://ethereum.org/en/developers/docs/nodes-and-clients/#client-diversity --internalcl it's the way to run our CL (Caplin) inside Erigon's process.

AskAlexSharov commented 4 months ago

@RickyEsclapon did you use https://erigon.gitbook.io/ ?

erigontech / erigon