Staged Sync err="[1/15 Snapshots] BuildMissedIndices: ParseTransaction & CPU pinned at 100%, no progress

CatspersCoffee commented 1 year ago

System information

Erigon version: 2_48_1-stable-674b77f0

OS & Version:

Ubuntu 22.04.3 LTS 5.15.0-83-generic

Commit hash: 674b77f0

Erigon Command (with flags/config):

/erigon \
      --datadir /mnt/data1/data/mainnet/erigon_2_48_0_TEST4 \
      --chain=mainnet \
      --port=30303 \
      --private.api.addr=0.0.0.0:9090 \
      --authrpc.addr="0.0.0.0" \
      --authrpc.port=8551 \
      --authrpc.vhosts="0.0.0.0" \
      --authrpc.jwtsecret=/mnt/data1/secrets/jwtsecret \
      --torrent.port=42069 \
      --port 6060 \
      --prune htc \
      --metrics \
      --pprof \
      --http \
      --http.port=8545 \
      --http.addr="0.0.0.0" \
      --ws \
      --ws.compression \
      --db.read.concurrency 80 \
      --rpc.batch.concurrency 80 \
      --rpc.batch.limit 80000 \
      --http.api="eth,erigon,admin,debug,net,trace,txpool,web3,erigon"

Concensus Layer: Prysm 4.0.8

Concensus Layer Command (with flags/config):

./beacon-chain \
        --mainnet \
        --datadir=/mnt/data1/data/mainnet/prysm/beacon_4_0_8_TEST8 \
        --checkpoint-sync-url=https://beaconstate.info \
        --genesis-beacon-api-url=https://beaconstate.info \
        --execution-endpoint http://127.0.0.1:8551 \
        --jwt-secret=/mnt/data1/secrets/jwtsecret \
        --accept-terms-of-use=true \
        --enable-optional-engine-methods \
        --suggested-fee-recipient="0xC...<removed>...9"

Chain/Network: Mainnet

Expected behaviour

Full sync from clean directories. Expected to move from Download Snapshots Stage --> Download Headers Stage

Actual behaviour

Erigon goes around in circles with 100% CPU.

INFO[09-07|10:36:55.354] [1/15 Snapshots] Fetching torrent files metadata 
INFO[09-07|10:36:56.751] [snapshots] Blocks Stat                  blocks=9500k indices=9500k alloc=13.6GB sys=19.3GB
INFO[09-07|10:37:35.195] [p2p] GoodPeers                          eth67=16 eth68=9 eth66=6
INFO[09-07|10:37:35.208] [txpool] stat                            pending=237 baseFee=0 queued=2631 alloc=10.8GB sys=19.3GB
EROR[09-07|10:38:35.136] Staged Sync                              err="[1/15 Snapshots] BuildMissedIndices: ParseTransaction: rlp parse transaction: R: rlp parse: integer encoding for RLP must not have leading zeros: 00, blockNum: 13054679, i: 10280749"
INFO[09-07|10:38:35.207] [txpool] stat                            pending=237 baseFee=0 queued=2631 alloc=11.6GB sys=19.3GB
INFO[09-07|10:38:35.638] [1/15 Snapshots] Fetching torrent files metadata 
INFO[09-07|10:38:37.420] [snapshots] Blocks Stat                  blocks=9500k indices=9500k alloc=3.6GB sys=19.4GB
INFO[09-07|10:39:35.195] [p2p] GoodPeers                          eth68=12 eth66=5 eth67=15
INFO[09-07|10:39:35.208] [txpool] stat                            pending=237 baseFee=0 queued=2631 alloc=10.5GB sys=19.4GB
EROR[09-07|10:40:13.890] Staged Sync                              err="[1/15 Snapshots] BuildMissedIndices: ParseTransaction: rlp parse transaction: R: rlp parse: integer encoding for RLP must not have leading zeros: 00, blockNum: 13054679, i: 10280749"
INFO[09-07|10:40:14.391] [1/15 Snapshots] Fetching torrent files metadata 
INFO[09-07|10:40:16.158] [snapshots] Blocks Stat                  blocks=9500k indices=9500k alloc=3.1GB sys=19.4GB
INFO[09-07|10:40:35.206] [txpool] stat                            pending=237 baseFee=0 queued=2631 alloc=7.6GB sys=19.4GB
INFO[09-07|10:41:35.196] [p2p] GoodPeers                          eth68=11 eth67=15 eth66=4
INFO[09-07|10:41:35.208] [txpool] stat                            pending=237 baseFee=0 queued=2631 alloc=11.1GB sys=19.4GB
EROR[09-07|10:41:52.883] Staged Sync                              err="[1/15 Snapshots] BuildMissedIndices: ParseTransaction: rlp parse transaction: R: rlp parse: integer encoding for RLP must not have leading zeros: 00, blockNum: 13054679, i: 10280749"
INFO[09-07|10:41:53.383] [1/15 Snapshots] Fetching torrent files metadata 
INFO[09-07|10:41:55.070] [snapshots] Blocks Stat                  blocks=9500k indices=9500k alloc=5.2GB sys=19.4GB
INFO[09-07|10:42:35.207] [txpool] stat                            pending=237 baseFee=0 queued=2631 alloc=8.7GB sys=19.4GB
EROR[09-07|10:43:32.057] Staged Sync                              err="[1/15 Snapshots] BuildMissedIndices: ParseTransaction: rlp parse transaction: R: rlp parse: integer encoding for RLP must not have leading zeros: 00, blockNum: 13054679, i: 10280749"

Steps to reproduce the behaviour

Run start script with cl options above. leave to download snapshots, verify then move to next stage, 2hr+ no change in output on the logs. CPU is at 100% on 12 threads. All drives are NVMe. This is from a fresh sync with clean data directories. All erigon and consensus related ports open on firewall.

[1/15 Snapshots] download move through to Verify completes 100% Then the above error/output happens with no change after a significant amount of time.

AskAlexSharov commented 1 year ago

start erigon once with --downloader.verify flag

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 40 days with no activity. Remove stale label or comment, or this will be closed in 7 days.

github-actions[bot] commented 12 months ago

This issue was closed because it has been stalled for 7 days with no activity.

erigontech / erigon