search

testinprod-io / op-erigon

Optimism implementation on the efficiency frontier
https://op-erigon.testinprod.io
GNU Lesser General Public License v3.0
87 stars 15 forks source link

Very slow sync issue #199

Closed andreclaro closed 4 months ago

andreclaro commented 4 months ago

Description

I started a optimism mainnet node with op-erigon and op-node using the latest snapshot (op-mainnet-db-121527384.zst), however the node is very slowly syncing.

We are using an ec2 instance m6a.4xlarge (16 cpu cores and 64 gb memory) with io2 ebs volume with 30k iops.

I don't see any errors in the logs.

Syncing speed: < 1 block per second

[2024-07-01 10:22:53] Latest Block Number: 121630486
[2024-07-01 10:23:53] Latest Block Number: 121630557
[2024-07-01 10:24:53] Latest Block Number: 121630607
[2024-07-01 10:25:53] Latest Block Number: 121630669
[2024-07-01 10:26:53] Latest Block Number: 121630694
[2024-07-01 10:27:53] Latest Block Number: 121630734
[2024-07-01 10:28:53] Latest Block Number: 121630756

System information

Erigon version: v2.60.1-0.6.4

erigon --version
erigon version 0.06.4-unstable

OS & Version: Ubuntu (Linux)

Erigon Command (with flags/config):

/usr/local/bin/erigon \
    --chain op-mainnet \
    --datadir /var/lib/optimism/data \
    --snapshots=false \
    --torrent.port 42069 \
    --torrent.download.rate 1000mb \
    --port 30303 \
    --http \
    --http.addr 0.0.0.0 \
    --http.port 8545 \
    --http.vhosts "*" \
    --http.api=web3,debug,eth,net,engine,erigon \
    --ws \
    --ws.port 8546 \
    --authrpc.addr 127.0.0.1 \
    --authrpc.vhosts="*" \
    --authrpc.port 8551 \
    --authrpc.jwtsecret /var/lib/optimism/jwt.txt \
    --rollup.sequencerhttp https://mainnet-sequencer.optimism.io \
    --db.size.limit 8TB \
    --txpool.gossip.disable=true \
    --nodiscover \
    --p2p.allowed-ports=46303,46304,46305 \
    --p2p.protocol=66,67,68 \
    --metrics \
    --metrics.addr 0.0.0.0 \
    --metrics.port=6060 \
    --verbosity 3 \
    --log.dir.disable

Op-node:

/usr/local/bin/op-node \
    --l1.trustrpc \
    --l1.rpckind=basic \
    --l1=http://L1_ADDRESS:8545 \
    --l1.beacon=http://L1_ADDRESS:5052 \
    --l2=http://127.0.0.1:8551 \
    --rpc.addr=127.0.0.1 \
    --rpc.port=9545 \
    --l2.jwt-secret=/var/lib/optimism/jwt.txt \
    --network=op-mainnet \
    --p2p.peerstore.path=/var/lib/optimism/data/opnode_peerstore_db \
    --p2p.priv.path=/var/lib/optimism/data/opnode_p2p_priv.txt \
    --p2p.discovery.path=/var/lib/optimism/data/opnode_discovery_db \
    --rollup.load-protocol-versions=true \
    --rollup.halt=major

Chain/Network: Optimism Mainnet

Expected behaviour

Sync quickly to the latest block.

Actual behaviour

It is not catching up with the public network.

Logs

op-erigon:

Jul 01 09:10:22 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:22.584] Built block                              hash=0x54bd5bd5ecd5c775fbd6a42ec6829b6f0d1eb172a77f9787929515aa6c3a8962 height=121626872 txs=11 gas used %=5.074 time=115.079556ms
Jul 01 09:10:22 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:22.590] [NewPayload] Handling new payload        height=121626872 hash=0x54bd5bd5ecd5c775fbd6a42ec6829b6f0d1eb172a77f9787929515aa6c3a8962
Jul 01 09:10:22 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:22.609] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
Jul 01 09:10:23 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:23.163] RPC Daemon notified of new headers       from=121626871 to=121626872 amount=1 hash=0x54bd5bd5ecd5c775fbd6a42ec6829b6f0d1eb172a77f9787929515aa6c3a8962 header sending=20.38µs log sending=230ns
Jul 01 09:10:23 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:23.163] head updated                             hash=0x54bd5bd5ecd5c775fbd6a42ec6829b6f0d1eb172a77f9787929515aa6c3a8962 number=121626872
Jul 01 09:10:23 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:23.169] [ForkChoiceUpdated] BlockBuilder added   payload=98806
Jul 01 09:10:23 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:23.169] Building block...
Jul 01 09:10:23 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:23.169] Received GetPayloadV3                    payloadId=98806
Jul 01 09:10:23 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:23.171] stage running - force txs, with params   txs=6 notxpool=true
Jul 01 09:10:23 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:23.171] [2/7 MiningCreateBlock] Start mine       block=121626873 baseFee=60251063 gasLimit=30000000
Jul 01 09:10:23 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:23.222] Built block                              hash=0x33e5aa251435b5042f8ba80f0d3d8cec5e5058355cac4f410ce04dddaf742550 height=121626873 txs=6 gas used %=4.001 time=52.62547ms
Jul 01 09:10:23 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:23.226] [NewPayload] Handling new payload        height=121626873 hash=0x33e5aa251435b5042f8ba80f0d3d8cec5e5058355cac4f410ce04dddaf742550
Jul 01 09:10:23 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:23.248] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
Jul 01 09:10:23 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:23.625] RPC Daemon notified of new headers       from=121626872 to=121626873 amount=1 hash=0x33e5aa251435b5042f8ba80f0d3d8cec5e5058355cac4f410ce04dddaf742550 header sending=22.44µs log sending=210ns
Jul 01 09:10:23 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:23.625] head updated                             hash=0x33e5aa251435b5042f8ba80f0d3d8cec5e5058355cac4f410ce04dddaf742550 number=121626873
Jul 01 09:10:23 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:23.630] [ForkChoiceUpdated] BlockBuilder added   payload=98807
Jul 01 09:10:23 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:23.630] Building block...
Jul 01 09:10:23 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:23.631] Received GetPayloadV3                    payloadId=98807
Jul 01 09:10:23 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:23.632] stage running - force txs, with params   txs=11 notxpool=true
Jul 01 09:10:23 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:23.633] [2/7 MiningCreateBlock] Start mine       block=121626874 baseFee=60067911 gasLimit=30000000
Jul 01 09:10:23 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:23.729] Built block                              hash=0xab97f27615356986a2e4385f44f0a5e5b2c943c61cbdfaa949b1d03633db4549 height=121626874 txs=11 gas used %=7.375 time=98.220291ms
Jul 01 09:10:23 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:23.733] [NewPayload] Handling new payload        height=121626874 hash=0xab97f27615356986a2e4385f44f0a5e5b2c943c61cbdfaa949b1d03633db4549
Jul 01 09:10:23 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:23.766] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
Jul 01 09:10:24 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:24.263] RPC Daemon notified of new headers       from=121626873 to=121626874 amount=1 hash=0xab97f27615356986a2e4385f44f0a5e5b2c943c61cbdfaa949b1d03633db4549 header sending=21µs log sending=360ns
Jul 01 09:10:24 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:24.263] head updated                             hash=0xab97f27615356986a2e4385f44f0a5e5b2c943c61cbdfaa949b1d03633db4549 number=121626874
Jul 01 09:10:24 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:24.270] [ForkChoiceUpdated] BlockBuilder added   payload=98808
Jul 01 09:10:24 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:24.270] Building block...
Jul 01 09:10:24 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:24.270] Received GetPayloadV3                    payloadId=98808
Jul 01 09:10:24 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:24.272] stage running - force txs, with params   txs=14 notxpool=true
Jul 01 09:10:24 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:24.272] [2/7 MiningCreateBlock] Start mine       block=121626875 baseFee=59933965 gasLimit=30000000
Jul 01 09:10:25 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:25.115] Built block                              hash=0x672def350a5226f0276eebcdba8bd20ece0f420f832fb3cbc15c505cabb6731e height=121626875 txs=14 gas used %=89.711 time=845.040732ms
Jul 01 09:10:25 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:25.124] [NewPayload] Handling new payload        height=121626875 hash=0x672def350a5226f0276eebcdba8bd20ece0f420f832fb3cbc15c505cabb6731e
Jul 01 09:10:25 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:25.524] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
Jul 01 09:10:26 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:26.738] RPC Daemon notified of new headers       from=121626874 to=121626875 amount=1 hash=0x672def350a5226f0276eebcdba8bd20ece0f420f832fb3cbc15c505cabb6731e header sending=31.47µs log sending=260ns
Jul 01 09:10:26 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:26.738] head updated                             hash=0x672def350a5226f0276eebcdba8bd20ece0f420f832fb3cbc15c505cabb6731e number=121626875
Jul 01 09:10:26 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:26.747] [ForkChoiceUpdated] BlockBuilder added   payload=98809
Jul 01 09:10:26 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:26.747] Building block...
Jul 01 09:10:26 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:26.747] Received GetPayloadV3                    payloadId=98809
Jul 01 09:10:26 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:26.750] stage running - force txs, with params   txs=16 notxpool=true
Jul 01 09:10:26 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:26.750] [2/7 MiningCreateBlock] Start mine       block=121626876 baseFee=60984640 gasLimit=30000000
Jul 01 09:10:26 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:26.958] Built block                              hash=0x8f180f73a0253085e71a22feb4b9c7a7deb4fbe17be4e8ead24fd125d626b549 height=121626876 txs=16 gas used %=7.541 time=210.508638ms
Jul 01 09:10:26 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:26.962] [NewPayload] Handling new payload        height=121626876 hash=0x8f180f73a0253085e71a22feb4b9c7a7deb4fbe17be4e8ead24fd125d626b549
Jul 01 09:10:26 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:26.993] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
Jul 01 09:10:27 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:27.821] RPC Daemon notified of new headers       from=121626875 to=121626876 amount=1 hash=0x8f180f73a0253085e71a22feb4b9c7a7deb4fbe17be4e8ead24fd125d626b549 header sending=21.14µs log sending=240ns
Jul 01 09:10:27 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:27.822] head updated                             hash=0x8f180f73a0253085e71a22feb4b9c7a7deb4fbe17be4e8ead24fd125d626b549 number=121626876
Jul 01 09:10:27 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:27.827] [ForkChoiceUpdated] BlockBuilder added   payload=98810
Jul 01 09:10:27 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:27.827] Building block...
Jul 01 09:10:27 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:27.827] Received GetPayloadV3                    payloadId=98810
Jul 01 09:10:27 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:27.829] stage running - force txs, with params   txs=15 notxpool=true
Jul 01 09:10:27 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:27.829] [2/7 MiningCreateBlock] Start mine       block=121626877 baseFee=60851074 gasLimit=30000000
Jul 01 09:10:27 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:27.994] Built block                              hash=0x3d8ae0a5b9f9048191fb7f75059bd6a0a0ef00f190f26066ca8a614f5db81882 height=121626877 txs=15 gas used %=4.547 time=167.084472ms
Jul 01 09:10:27 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:27.999] [NewPayload] Handling new payload        height=121626877 hash=0x3d8ae0a5b9f9048191fb7f75059bd6a0a0ef00f190f26066ca8a614f5db81882
Jul 01 09:10:28 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:28.021] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
Jul 01 09:10:28 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:28.869] RPC Daemon notified of new headers       from=121626876 to=121626877 amount=1 hash=0x3d8ae0a5b9f9048191fb7f75059bd6a0a0ef00f190f26066ca8a614f5db81882 header sending=20.57µs log sending=260ns
Jul 01 09:10:28 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:28.869] head updated                             hash=0x3d8ae0a5b9f9048191fb7f75059bd6a0a0ef00f190f26066ca8a614f5db81882 number=121626877
Jul 01 09:10:28 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:28.874] [ForkChoiceUpdated] BlockBuilder added   payload=98811
Jul 01 09:10:28 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:28.874] Building block...
Jul 01 09:10:28 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:28.874] Received GetPayloadV3                    payloadId=98811
Jul 01 09:10:28 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:28.876] stage running - force txs, with params   txs=6 notxpool=true
Jul 01 09:10:28 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:28.876] [2/7 MiningCreateBlock] Start mine       block=121626878 baseFee=60674079 gasLimit=30000000
Jul 01 09:10:29 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:29.055] Built block                              hash=0x4a3d1a89cd3fe8b11e71d43770b991d708db5ab7225cf04b80f1eb9b649710fd height=121626878 txs=6 gas used %=2.785 time=181.42694ms
Jul 01 09:10:29 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:29.060] [NewPayload] Handling new payload        height=121626878 hash=0x4a3d1a89cd3fe8b11e71d43770b991d708db5ab7225cf04b80f1eb9b649710fd
Jul 01 09:10:29 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:29.077] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
Jul 01 09:10:29 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:29.636] RPC Daemon notified of new headers       from=121626877 to=121626878 amount=1 hash=0x4a3d1a89cd3fe8b11e71d43770b991d708db5ab7225cf04b80f1eb9b649710fd header sending=19.71µs log sending=260ns
Jul 01 09:10:29 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:29.636] head updated                             hash=0x4a3d1a89cd3fe8b11e71d43770b991d708db5ab7225cf04b80f1eb9b649710fd number=121626878
Jul 01 09:10:29 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:29.647] [ForkChoiceUpdated] BlockBuilder added   payload=98812
Jul 01 09:10:29 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:29.647] Building block...
Jul 01 09:10:29 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:29.647] Received GetPayloadV3                    payloadId=98812
Jul 01 09:10:29 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:29.649] stage running - force txs, with params   txs=6 notxpool=true
Jul 01 09:10:29 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:29.649] [2/7 MiningCreateBlock] Start mine       block=121626879 baseFee=60471943 gasLimit=30000000
Jul 01 09:10:29 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:29.727] Built block                              hash=0x9c4891490a7560b32009581fa73a6734d2c337c2ab76546745694007b5012f59 height=121626879 txs=6 gas used %=1.586 time=80.323273ms
Jul 01 09:10:29 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:29.730] [NewPayload] Handling new payload        height=121626879 hash=0x9c4891490a7560b32009581fa73a6734d2c337c2ab76546745694007b5012f59
Jul 01 09:10:29 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:29.752] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
Jul 01 09:10:30 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:30.014] RPC Daemon notified of new headers       from=121626878 to=121626879 amount=1 hash=0x9c4891490a7560b32009581fa73a6734d2c337c2ab76546745694007b5012f59 header sending=21.44µs log sending=270ns
Jul 01 09:10:30 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:30.014] head updated                             hash=0x9c4891490a7560b32009581fa73a6734d2c337c2ab76546745694007b5012f59 number=121626879
Jul 01 09:10:30 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:30.022] [ForkChoiceUpdated] BlockBuilder added   payload=98813
Jul 01 09:10:30 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:30.022] Building block...
Jul 01 09:10:30 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:30.022] Received GetPayloadV3                    payloadId=98813
Jul 01 09:10:30 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:30.024] stage running - force txs, with params   txs=8 notxpool=true
Jul 01 09:10:30 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:30.024] [2/7 MiningCreateBlock] Start mine       block=121626880 baseFee=60253080 gasLimit=30000000
Jul 01 09:10:30 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:30.112] Built block                              hash=0xd98feec67c2e9374cdc126165cdc25999788780e4d17ce2beb29c0f2ba7eb017 height=121626880 txs=8 gas used %=3.034 time=90.077319ms
Jul 01 09:10:30 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:30.117] [NewPayload] Handling new payload        height=121626880 hash=0xd98feec67c2e9374cdc126165cdc25999788780e4d17ce2beb29c0f2ba7eb017
Jul 01 09:10:30 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:30.141] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
Jul 01 09:10:30 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:30.535] RPC Daemon notified of new headers       from=121626879 to=121626880 amount=1 hash=0xd98feec67c2e9374cdc126165cdc25999788780e4d17ce2beb29c0f2ba7eb017 header sending=19.37µs log sending=260ns
Jul 01 09:10:30 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:30.535] head updated                             hash=0xd98feec67c2e9374cdc126165cdc25999788780e4d17ce2beb29c0f2ba7eb017 number=121626880
Jul 01 09:10:30 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:30.541] [ForkChoiceUpdated] BlockBuilder added   payload=98814
Jul 01 09:10:30 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:30.541] Building block...
Jul 01 09:10:30 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:30.542] Received GetPayloadV3                    payloadId=98814
Jul 01 09:10:30 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:30.543] stage running - force txs, with params   txs=12 notxpool=true
Jul 01 09:10:30 m-optimism-03 erigon[873]: [INFO] [07-01|09:10:30.543] [2/7 MiningCreateBlock] Start mine       block=121626881 baseFee=60055936 gasLimit=30000000

op-node:

Jul 01 09:10:42 m-optimism-03 op-node[874]: t=2024-07-01T09:10:42+0000 lvl=info msg="Optimistically queueing unsafe L2 execution payload" id=0xb81f76f16f229dcf112edf6400b3fb6835aaf5707e0b278f3e7ce26019a5059e:122113132
Jul 01 09:10:42 m-optimism-03 op-node[874]: t=2024-07-01T09:10:42+0000 lvl=info msg="Dropping payload from payload queue because the payload queue is too large" id=0xfeaf0c3caa8638c5f005de4cd7ce0250a6759d90d7c826d5f66697895f82f544:122058943
Jul 01 09:10:42 m-optimism-03 op-node[874]: t=2024-07-01T09:10:42+0000 lvl=info msg="generated attributes in payload queue" txs=11 timestamp=1718852547
Jul 01 09:10:43 m-optimism-03 op-node[874]: t=2024-07-01T09:10:43+0000 lvl=info msg="inserted block" hash=0x6dd8673042430054baeb45bf4c83d8dda0f8a8f5fceb38b39aefbb8dd52505c1 number=121626885 state_root=0x5c5931e8c3e747282407a2ded4ea0427a9f353733a8b5417c5e1ad864c7ea677 timestamp=1718852547 parent=0x4ad5432dd6f73815fabac90b3b76e25e96d51e60c59714e5c56a1221ecc15318 prev_randao=0xf46a1d6afae56a60da4dfac2f225b72c44ddec8fb8c7e5117ad1ee60542568cc fee_recipient=0x4200000000000000000000000000000000000011 txs=11 update_safe=false
Jul 01 09:10:43 m-optimism-03 op-node[874]: t=2024-07-01T09:10:43+0000 lvl=info msg="generated attributes in payload queue" txs=11 timestamp=1718852549
Jul 01 09:10:44 m-optimism-03 op-node[874]: t=2024-07-01T09:10:44+0000 lvl=info msg="Received signed execution payload from p2p" id=0x99879029028522d02831df01e2e6a653487c7caa0f4e8afa0bb296bf09110e3c:122113133 peer=16Uiu2HAkxAeKS9un9jBrt5MyYbbX1dWe1Jnnwjo8wraioimQoA6v
Jul 01 09:10:44 m-optimism-03 op-node[874]: t=2024-07-01T09:10:44+0000 lvl=info msg="inserted block" hash=0x4316d848bdd407afda42025b0b616d3e575a95f25875a64dad871632336a67a0 number=121626886 state_root=0x75bcaefdafe90646fcd7deca40cee972a1af39b292132c746fc948696a75e4e9 timestamp=1718852549 parent=0x6dd8673042430054baeb45bf4c83d8dda0f8a8f5fceb38b39aefbb8dd52505c1 prev_randao=0xf46a1d6afae56a60da4dfac2f225b72c44ddec8fb8c7e5117ad1ee60542568cc fee_recipient=0x4200000000000000000000000000000000000011 txs=11 update_safe=false
Jul 01 09:10:44 m-optimism-03 op-node[874]: t=2024-07-01T09:10:44+0000 lvl=info msg="Optimistically queueing unsafe L2 execution payload" id=0x99879029028522d02831df01e2e6a653487c7caa0f4e8afa0bb296bf09110e3c:122113133
Jul 01 09:10:44 m-optimism-03 op-node[874]: t=2024-07-01T09:10:44+0000 lvl=info msg="Dropping payload from payload queue because the payload queue is too large" id=0xb2e26f06f1db417f2d95e13e39fcb7b38bef086a87d7237d3ea42af49f14d77e:122058944
Jul 01 09:10:44 m-optimism-03 op-node[874]: t=2024-07-01T09:10:44+0000 lvl=info msg="generated attributes in payload queue" txs=6 timestamp=1718852551
Jul 01 09:10:44 m-optimism-03 op-node[874]: t=2024-07-01T09:10:44+0000 lvl=info msg="inserted block" hash=0xbcfcd1bfd3cd501f317a9d5411de3b24ec10cb676acab0466b04742797e1b29b number=121626887 state_root=0x93c0682659d3ff6d44b1e619f5e6dc2b93543b92993e3439cc5066e29a91ba1b timestamp=1718852551 parent=0x4316d848bdd407afda42025b0b616d3e575a95f25875a64dad871632336a67a0 prev_randao=0xf46a1d6afae56a60da4dfac2f225b72c44ddec8fb8c7e5117ad1ee60542568cc fee_recipient=0x4200000000000000000000000000000000000011 txs=6 update_safe=false
Jul 01 09:10:44 m-optimism-03 op-node[874]: t=2024-07-01T09:10:44+0000 lvl=info msg="generated attributes in payload queue" txs=8 timestamp=1718852553
Jul 01 09:10:45 m-optimism-03 op-node[874]: t=2024-07-01T09:10:45+0000 lvl=info msg="inserted block" hash=0xa76da4ce92ca32c9e8da37280ffcc7430b23821c2379554c03e9e69523508169 number=121626888 state_root=0xb680a5eb9996b775bc87674267f3d1d3ba574337856fb61a31738fd9879eb3f4 timestamp=1718852553 parent=0xbcfcd1bfd3cd501f317a9d5411de3b24ec10cb676acab0466b04742797e1b29b prev_randao=0xf46a1d6afae56a60da4dfac2f225b72c44ddec8fb8c7e5117ad1ee60542568cc fee_recipient=0x4200000000000000000000000000000000000011 txs=8 update_safe=false
Jul 01 09:10:45 m-optimism-03 op-node[874]: t=2024-07-01T09:10:45+0000 lvl=info msg="generated attributes in payload queue" txs=12 timestamp=1718852555
Jul 01 09:10:45 m-optimism-03 op-node[874]: t=2024-07-01T09:10:45+0000 lvl=info msg="Received signed execution payload from p2p" id=0xa89dba2c6c97bedbbe6bb3ddb43679d4240242e35a56467a476bce26169e2990:122113134 peer=16Uiu2HAmTR8rgg7iCJCQzso4GiRwedSv176Dv8uMuDd5pBAc6T7z
Jul 01 09:10:46 m-optimism-03 op-node[874]: t=2024-07-01T09:10:46+0000 lvl=info msg="inserted block" hash=0xecdd07d49c6f8e9f1a6c007ece68ec092fd1b02fc49c215e2e4ad63cebb03655 number=121626889 state_root=0x2e400e75b63925df3ef3bab131c0d942d7167b2831563a07811d9d8678a88a93 timestamp=1718852555 parent=0xa76da4ce92ca32c9e8da37280ffcc7430b23821c2379554c03e9e69523508169 prev_randao=0xf46a1d6afae56a60da4dfac2f225b72c44ddec8fb8c7e5117ad1ee60542568cc fee_recipient=0x4200000000000000000000000000000000000011 txs=12 update_safe=false
Jul 01 09:10:46 m-optimism-03 op-node[874]: t=2024-07-01T09:10:46+0000 lvl=info msg="generated attributes in payload queue" txs=10 timestamp=1718852557
Jul 01 09:10:46 m-optimism-03 op-node[874]: t=2024-07-01T09:10:46+0000 lvl=info msg="inserted block" hash=0xf49fe4868565a5dac9780fe8ac50252961179a8600bc4aad080ac19831a450e0 number=121626890 state_root=0x8d899566b353bbf145b4fb8e2021bcfa8ee33d6dedbcc09c839ce8bfdd339b32 timestamp=1718852557 parent=0xecdd07d49c6f8e9f1a6c007ece68ec092fd1b02fc49c215e2e4ad63cebb03655 prev_randao=0x1d0684e7169fdef1fe6bcee32b9430b52c107607ac336f3db64a291a105f8176 fee_recipient=0x4200000000000000000000000000000000000011 txs=10 update_safe=false
Jul 01 09:10:46 m-optimism-03 op-node[874]: t=2024-07-01T09:10:46+0000 lvl=info msg="Optimistically queueing unsafe L2 execution payload" id=0xa89dba2c6c97bedbbe6bb3ddb43679d4240242e35a56467a476bce26169e2990:122113134
Jul 01 09:10:46 m-optimism-03 op-node[874]: t=2024-07-01T09:10:46+0000 lvl=info msg="Dropping payload from payload queue because the payload queue is too large" id=0x03e02c56e7a985f70f33c04984029e4f06c14f7e2fe354a5f3991d78fd9e64c8:122058945
Jul 01 09:10:46 m-optimism-03 op-node[874]: t=2024-07-01T09:10:46+0000 lvl=info msg="generated attributes in payload queue" txs=16 timestamp=1718852559
Jul 01 09:10:47 m-optimism-03 op-node[874]: t=2024-07-01T09:10:47+0000 lvl=info msg="Received signed execution payload from p2p" id=0x7f17255fc94f91b44c4462c869f1a9b9a1d24f6a4cb1b1ddf61c396a6e5cd5b2:122113135 peer=16Uiu2HAmDCQyvVr4jjyzmEN9ayhpfNs8J5RScmaYfjTJuoAydc1m
Jul 01 09:10:49 m-optimism-03 op-node[874]: t=2024-07-01T09:10:49+0000 lvl=info msg="Received signed execution payload from p2p" id=0x86a4b1fa897207ca732a431fcadb4657d965739326af66faa048a1f6ae149a93:122113136 peer=16Uiu2HAmRhsnEigxSwu4vDGed6h8AtWcGHTRmRf2jrYt33emZ5g8
Jul 01 09:10:49 m-optimism-03 op-node[874]: t=2024-07-01T09:10:49+0000 lvl=info msg="inserted block" hash=0xebee44492b7d22c9c1f1d7dfdfce3ba93347d3f6bae1fc53d67f3e0e78b8fe2d number=121626891 state_root=0x2627926712f14b938ccce0417ac5466573f38a41ead4db4732ab8711e65f76b1 timestamp=1718852559 parent=0xf49fe4868565a5dac9780fe8ac50252961179a8600bc4aad080ac19831a450e0 prev_randao=0x1d0684e7169fdef1fe6bcee32b9430b52c107607ac336f3db64a291a105f8176 fee_recipient=0x4200000000000000000000000000000000000011 txs=16 update_safe=false
Jul 01 09:10:49 m-optimism-03 op-node[874]: t=2024-07-01T09:10:49+0000 lvl=info msg="Optimistically queueing unsafe L2 execution payload" id=0x7f17255fc94f91b44c4462c869f1a9b9a1d24f6a4cb1b1ddf61c396a6e5cd5b2:122113135
Jul 01 09:10:49 m-optimism-03 op-node[874]: t=2024-07-01T09:10:49+0000 lvl=info msg="Dropping payload from payload queue because the payload queue is too large" id=0x9bd0cea01ff091a0eda4610faac182fd8a7f4fc5bddd218b1a301a9e1100707e:122058946
Jul 01 09:10:49 m-optimism-03 op-node[874]: t=2024-07-01T09:10:49+0000 lvl=info msg="Dropping payload from payload queue because the payload queue is too large" id=0xc3054d77acb41ce4b746f4ee0e43caf28c3715aadf9048b14dde4b4466bba79d:122058947
Jul 01 09:10:49 m-optimism-03 op-node[874]: t=2024-07-01T09:10:49+0000 lvl=info msg="Dropping payload from payload queue because the payload queue is too large" id=0x99c782c8b869f27e75c8db5a76948d89e90a571f10248c1c094e2489686622ce:122058948
Jul 01 09:10:49 m-optimism-03 op-node[874]: t=2024-07-01T09:10:49+0000 lvl=info msg="generated attributes in payload queue" txs=15 timestamp=1718852561
Jul 01 09:10:49 m-optimism-03 op-node[874]: t=2024-07-01T09:10:49+0000 lvl=info msg="Optimistically queueing unsafe L2 execution payload" id=0x86a4b1fa897207ca732a431fcadb4657d965739326af66faa048a1f6ae149a93:122113136
Jul 01 09:10:49 m-optimism-03 op-node[874]: t=2024-07-01T09:10:49+0000 lvl=info msg="Dropping payload from payload queue because the payload queue is too large" id=0x97cfb30e154ed7bcef7479a9e27c80afb4b0d54fbbd6d9f19e18707a83e5e599:122058949
Jul 01 09:10:50 m-optimism-03 op-node[874]: t=2024-07-01T09:10:50+0000 lvl=info msg="inserted block" hash=0x80db68c03d6a24bb1dc779de7806e1cfd8120947cf4dcc189508ed8d86dfb9a6 number=121626892 state_root=0x196b9eae85cec96be04cb38586461717f8b37ba40ac34c6d732335a06f764391 timestamp=1718852561 parent=0xebee44492b7d22c9c1f1d7dfdfce3ba93347d3f6bae1fc53d67f3e0e78b8fe2d prev_randao=0x1d0684e7169fdef1fe6bcee32b9430b52c107607ac336f3db64a291a105f8176 fee_recipient=0x4200000000000000000000000000000000000011 txs=15 update_safe=false
Jul 01 09:10:50 m-optimism-03 op-node[874]: t=2024-07-01T09:10:50+0000 lvl=info msg="generated attributes in payload queue" txs=7 timestamp=1718852563
Jul 01 09:10:50 m-optimism-03 op-node[874]: t=2024-07-01T09:10:50+0000 lvl=info msg="inserted block" hash=0x29dedb1fef83b0c2c27534f54f787008c2dd53efafb3419b9120842230acb55e number=121626893 state_root=0x5ba0430b81556ef48064c70032b75612963b02b65442ae5974b9e026bd3ae5bc timestamp=1718852563 parent=0x80db68c03d6a24bb1dc779de7806e1cfd8120947cf4dcc189508ed8d86dfb9a6 prev_randao=0x1d0684e7169fdef1fe6bcee32b9430b52c107607ac336f3db64a291a105f8176 fee_recipient=0x4200000000000000000000000000000000000011 txs=7 update_safe=false
Jul 01 09:10:50 m-optimism-03 op-node[874]: t=2024-07-01T09:10:50+0000 lvl=info msg="generated attributes in payload queue" txs=12 timestamp=1718852565
Jul 01 09:10:51 m-optimism-03 op-node[874]: t=2024-07-01T09:10:51+0000 lvl=info msg="Received signed execution payload from p2p" id=0xd1c588d5a35a1dca437f502c133a52a087c7e3347c85feb7536631258897aa2c:122113137 peer=16Uiu2HAmHqjvRMcyS5SSpayaMmgT6Xw5xdbnMogDNzakaff23T1T
Jul 01 09:10:52 m-optimism-03 op-node[874]: t=2024-07-01T09:10:52+0000 lvl=info msg="inserted block" hash=0x99bff7ea8782ec4955a22620b336cb0a146e4acb3cf3007ee488180c5ebc4693 number=121626894 state_root=0x7db06bb813ab6e5f22a9a95dee226eedd78f61634c980e05e1406545c5080068 timestamp=1718852565 parent=0x29dedb1fef83b0c2c27534f54f787008c2dd53efafb3419b9120842230acb55e prev_randao=0x1d0684e7169fdef1fe6bcee32b9430b52c107607ac336f3db64a291a105f8176 fee_recipient=0x4200000000000000000000000000000000000011 txs=12 update_safe=false
Jul 01 09:10:52 m-optimism-03 op-node[874]: t=2024-07-01T09:10:52+0000 lvl=info msg="generated attributes in payload queue" txs=9 timestamp=1718852567
Jul 01 09:10:52 m-optimism-03 op-node[874]: t=2024-07-01T09:10:52+0000 lvl=info msg="Optimistically queueing unsafe L2 execution payload" id=0xd1c588d5a35a1dca437f502c133a52a087c7e3347c85feb7536631258897aa2c:122113137
Jul 01 09:10:52 m-optimism-03 op-node[874]: t=2024-07-01T09:10:52+0000 lvl=info msg="Dropping payload from payload queue because the payload queue is too large" id=0xe9bdcb1f7f7c1b742606c5a67a240e89d0b85d529844b065c3a592dc2ea1401b:122058950
Jul 01 09:10:52 m-optimism-03 op-node[874]: t=2024-07-01T09:10:52+0000 lvl=info msg="inserted block" hash=0xb99271a0e47c3d10ce74ebe2606b6090f3a733cf70d2e6d1846b211367edee1a number=121626895 state_root=0xfc50fae7f9b870ce3ac50b0b4bf9beb37bd6c32874b4008ee75e8e4017fc6f82 timestamp=1718852567 parent=0x99bff7ea8782ec4955a22620b336cb0a146e4acb3cf3007ee488180c5ebc4693 prev_randao=0x1d0684e7169fdef1fe6bcee32b9430b52c107607ac336f3db64a291a105f8176 fee_recipient=0x4200000000000000000000000000000000000011 txs=9 update_safe=false
Jul 01 09:10:52 m-optimism-03 op-node[874]: t=2024-07-01T09:10:52+0000 lvl=info msg="generated attributes in payload queue" txs=11 timestamp=1718852569
Jul 01 09:10:53 m-optimism-03 op-node[874]: t=2024-07-01T09:10:53+0000 lvl=info msg="Received signed execution payload from p2p" id=0x9018fbd2864153d2943708b816c68bb85918ce138d1c7dc1ef628ba43dada203:122113138 peer=16Uiu2HAmRhsnEigxSwu4vDGed6h8AtWcGHTRmRf2jrYt33emZ5g8
Jul 01 09:10:54 m-optimism-03 op-node[874]: t=2024-07-01T09:10:54+0000 lvl=info msg="inserted block" hash=0xc28fe625442034757b10fd9a96ef8eb2916aba0b24312a2cdd63cc59b598ded4 number=121626896 state_root=0xaf911bcdd48edfd87d9667d382c95633613ec1561be0d3dfda1327c17926522e timestamp=1718852569 parent=0xb99271a0e47c3d10ce74ebe2606b6090f3a733cf70d2e6d1846b211367edee1a prev_randao=0x66a0b2880bf135a6271b0f95482c9dff871d0fa9e7c163c3068dbf57d3c32e63 fee_recipient=0x4200000000000000000000000000000000000011 txs=11 update_safe=false
Jul 01 09:10:54 m-optimism-03 op-node[874]: t=2024-07-01T09:10:54+0000 lvl=info msg="Optimistically queueing unsafe L2 execution payload" id=0x9018fbd2864153d2943708b816c68bb85918ce138d1c7dc1ef628ba43dada203:122113138
Jul 01 09:10:54 m-optimism-03 op-node[874]: t=2024-07-01T09:10:54+0000 lvl=info msg="Dropping payload from payload queue because the payload queue is too large" id=0x22db026b4eb2222e1d30e82dfb1552c03ee97a7bbd6ce0cdfbbcf26899ca6b4d:122058951
Jul 01 09:10:54 m-optimism-03 op-node[874]: t=2024-07-01T09:10:54+0000 lvl=info msg="Dropping payload from payload queue because the payload queue is too large" id=0x5cf6a7daad89400793f62f4a80fdbd2cf229edc6929b4c39d98f8772fbc555a2:122058952
Jul 01 09:10:54 m-optimism-03 op-node[874]: t=2024-07-01T09:10:54+0000 lvl=info msg="Dropping payload from payload queue because the payload queue is too large" id=0xfd473d8779fbef1a7330986685dfa1cc759a18ed30faaa0dc6f25e583763bb46:122058953
Jul 01 09:10:54 m-optimism-03 op-node[874]: t=2024-07-01T09:10:54+0000 lvl=info msg="generated attributes in payload queue" txs=9 timestamp=1718852571
Jul 01 09:10:55 m-optimism-03 op-node[874]: t=2024-07-01T09:10:55+0000 lvl=info msg="inserted block" hash=0xa154923c1bd6ad021d7f9a1dc01adef300f0d2fad2a67041b6bc4ce0b413f5ba number=121626897 state_root=0xf7b9ad4ddc2db6cb7de87f1164d585095faae6f692d4c8952c4c76ed798e2def timestamp=1718852571 parent=0xc28fe625442034757b10fd9a96ef8eb2916aba0b24312a2cdd63cc59b598ded4 prev_randao=0x66a0b2880bf135a6271b0f95482c9dff871d0fa9e7c163c3068dbf57d3c32e63 fee_recipient=0x4200000000000000000000000000000000000011 txs=9 update_safe=false
Jul 01 09:10:55 m-optimism-03 op-node[874]: t=2024-07-01T09:10:55+0000 lvl=info msg="generated attributes in payload queue" txs=6 timestamp=1718852573
Jul 01 09:10:55 m-optimism-03 op-node[874]: t=2024-07-01T09:10:55+0000 lvl=info msg="inserted block" hash=0x0d70a9161dece03988b82fe28882579d93131ed4dab0f5186e2aff3fbef8a92d number=121626898 state_root=0xea194a0ba3872637e98e53a444b1abe9808a21ba57748ee85f3c58175f31dd68 timestamp=1718852573 parent=0xa154923c1bd6ad021d7f9a1dc01adef300f0d2fad2a67041b6bc4ce0b413f5ba prev_randao=0x66a0b2880bf135a6271b0f95482c9dff871d0fa9e7c163c3068dbf57d3c32e63 fee_recipient=0x4200000000000000000000000000000000000011 txs=6 update_safe=false
Jul 01 09:10:55 m-optimism-03 op-node[874]: t=2024-07-01T09:10:55+0000 lvl=info msg="generated attributes in payload queue" txs=7 timestamp=1718852575
Jul 01 09:10:55 m-optimism-03 op-node[874]: t=2024-07-01T09:10:55+0000 lvl=info msg="inserted block" hash=0xd9caa0fbc577ae37d0cee47f60b1f6d31e90a05023ca4d91e84a2a467da64010 number=121626899 state_root=0x250c089b97ab7687054ab478e34c3c404411bc20be9b5224d6a20d7a5e1eb182 timestamp=1718852575 parent=0x0d70a9161dece03988b82fe28882579d93131ed4dab0f5186e2aff3fbef8a92d prev_randao=0x66a0b2880bf135a6271b0f95482c9dff871d0fa9e7c163c3068dbf57d3c32e63 fee_recipient=0x4200000000000000000000000000000000000011 txs=7 update_safe=false
andreclaro commented 4 months ago

Let me know if you need more details, logs, etc. I will have the node running for more one or two days and then I will turn it down.

andreclaro commented 4 months ago

Using the latest release of op-node: https://github.com/ethereum-optimism/optimism/releases/tag/v1.7.7

mininny commented 4 months ago

Hi @andreclaro, it seems like the erigon node is building each block from op-node's ForkChoiceUpdate signal, which can slow down the sync speed. It might catch up to speed after a while, how's the syncing speed now?

Also, I believe you mis-linked the erigon snapshot in the issue description (it's linked to base-mainnet when it should be op-mainnet). Anyway, we also have more recent version of the erigon snapshot here, could you start from this snapshot as it would require little catchup to the chain head? https://datadirs.testinprod.io/op-mainnet-db-122131967.zst

andreclaro commented 4 months ago

Using the latest release of op-node: https://github.com/ethereum-optimism/optimism/releases/tag/v1.7.7

" it seems like the erigon node is building each block from op-node's ForkChoiceUpdate signal" ... What else could I do to speed up the sync?

The current speed is around 0.6 blocks per second ( (121707178-121707138)/60 )

[2024-07-02 15:37:27] Latest Block Number: 121707138
[2024-07-02 15:38:27] Latest Block Number: 121707178

Anyway, I will try using the latest snapshot (https://datadirs.testinprod.io/op-mainnet-db-122131967.zst), but it will be nice to understand what's wrong and/or how to improve the sync speed...

andreclaro commented 4 months ago

@mininny, using the latest snapshot is also syncing at a very very slow pace:

[2024-07-02 18:50:42] Safe L2 Number: 122138921
[2024-07-02 18:50:53] Safe L2 Number: 122138921
[2024-07-02 18:51:03] Safe L2 Number: 122138921
[2024-07-02 18:51:13] Safe L2 Number: 122138921
[2024-07-02 18:51:23] Safe L2 Number: 122139032
[2024-07-02 18:51:34] Safe L2 Number: 122139032
[2024-07-02 18:51:46] Safe L2 Number: 122139032
[2024-07-02 18:51:58] Safe L2 Number: 122139032
[2024-07-02 18:52:09] Safe L2 Number: 122139032
[2024-07-02 18:52:20] Safe L2 Number: 122139032
[2024-07-02 18:52:31] Safe L2 Number: 122139032
[2024-07-02 18:52:45] Safe L2 Number: 122139032
[2024-07-02 18:52:57] Safe L2 Number: 122139032
[2024-07-02 18:53:07] Safe L2 Number: 122139032
[2024-07-02 18:53:17] Safe L2 Number: 122139141
[2024-07-02 18:53:27] Safe L2 Number: 122139141
[2024-07-02 18:53:38] Safe L2 Number: 122139141

op-erigon:

Jul 02 18:54:24 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:24.451] RPC Daemon notified of new headers       from=122139208 to=122139209 amount=1 hash=0x6de7423fe1376a6800b4a273ae90904ab634d03f4027b86973c62c8fedc158f3 header sending=22.05µs log sending=360ns
Jul 02 18:54:24 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:24.451] head updated                             hash=0x6de7423fe1376a6800b4a273ae90904ab634d03f4027b86973c62c8fedc158f3 number=122139209
Jul 02 18:54:24 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:24.459] [ForkChoiceUpdated] BlockBuilder added   payload=7243
Jul 02 18:54:24 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:24.459] Building block...
Jul 02 18:54:24 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:24.460] Received GetPayloadV3                    payloadId=7243
Jul 02 18:54:24 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:24.463] stage running - force txs, with params   txs=9 notxpool=true
Jul 02 18:54:24 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:24.463] [2/7 MiningCreateBlock] Start mine       block=122139210 baseFee=59312343 gasLimit=30000000
Jul 02 18:54:25 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:25.476] Built block                              hash=0x03acdf17c57510d65f6f04c8a3cb6010dcf288b016725712c018452e26ee8b90 height=122139210 txs=9 gas used %=91.838 time=1.016951414s
Jul 02 18:54:25 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:25.487] [NewPayload] Handling new payload        height=122139210 hash=0x03acdf17c57510d65f6f04c8a3cb6010dcf288b016725712c018452e26ee8b90
Jul 02 18:54:25 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:25.905] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
Jul 02 18:54:27 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:27.472] RPC Daemon notified of new headers       from=122139209 to=122139210 amount=1 hash=0x03acdf17c57510d65f6f04c8a3cb6010dcf288b016725712c018452e26ee8b90 header sending=21.91µs log sending=380ns
Jul 02 18:54:27 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:27.472] head updated                             hash=0x03acdf17c57510d65f6f04c8a3cb6010dcf288b016725712c018452e26ee8b90 number=122139210
Jul 02 18:54:27 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:27.482] [ForkChoiceUpdated] BlockBuilder added   payload=7244
Jul 02 18:54:27 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:27.482] Building block...
Jul 02 18:54:27 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:27.482] Received GetPayloadV3                    payloadId=7244
Jul 02 18:54:27 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:27.486] stage running - force txs, with params   txs=11 notxpool=true
Jul 02 18:54:27 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:27.486] [2/7 MiningCreateBlock] Start mine       block=122139211 baseFee=60382397 gasLimit=30000000
Jul 02 18:54:27 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:27.669] Built block                              hash=0xb1f6513f7fb24682b1835f05eaeb27f8650cc7252107c1eb4bf28032e90269f9 height=122139211 txs=11 gas used %=5.904 time=186.891451ms
Jul 02 18:54:27 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:27.674] [NewPayload] Handling new payload        height=122139211 hash=0xb1f6513f7fb24682b1835f05eaeb27f8650cc7252107c1eb4bf28032e90269f9
Jul 02 18:54:27 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:27.703] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
Jul 02 18:54:28 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:28.535] RPC Daemon notified of new headers       from=122139210 to=122139211 amount=1 hash=0xb1f6513f7fb24682b1835f05eaeb27f8650cc7252107c1eb4bf28032e90269f9 header sending=28.08µs log sending=390ns
Jul 02 18:54:28 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:28.535] head updated                             hash=0xb1f6513f7fb24682b1835f05eaeb27f8650cc7252107c1eb4bf28032e90269f9 number=122139211
Jul 02 18:54:28 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:28.542] [ForkChoiceUpdated] BlockBuilder added   payload=7245
Jul 02 18:54:28 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:28.542] Building block...
Jul 02 18:54:28 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:28.542] Received GetPayloadV3                    payloadId=7245
Jul 02 18:54:28 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:28.545] stage running - force txs, with params   txs=4 notxpool=true
Jul 02 18:54:28 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:28.545] [2/7 MiningCreateBlock] Start mine       block=122139212 baseFee=60226427 gasLimit=30000000
Jul 02 18:54:28 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:28.575] Built block                              hash=0xa91a8eb3fd84fc199dbadfe87588433dd34c8454e1fea9a0676aa9417c9676d0 height=122139212 txs=4 gas used %=1.287 time=33.023082ms
Jul 02 18:54:28 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:28.578] [NewPayload] Handling new payload        height=122139212 hash=0xa91a8eb3fd84fc199dbadfe87588433dd34c8454e1fea9a0676aa9417c9676d0
Jul 02 18:54:28 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:28.601] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
Jul 02 18:54:28 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:28.827] RPC Daemon notified of new headers       from=122139211 to=122139212 amount=1 hash=0xa91a8eb3fd84fc199dbadfe87588433dd34c8454e1fea9a0676aa9417c9676d0 header sending=24.48µs log sending=390ns
Jul 02 18:54:28 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:28.827] head updated                             hash=0xa91a8eb3fd84fc199dbadfe87588433dd34c8454e1fea9a0676aa9417c9676d0 number=122139212
Jul 02 18:54:28 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:28.833] [ForkChoiceUpdated] BlockBuilder added   payload=7246
Jul 02 18:54:28 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:28.833] Building block...
Jul 02 18:54:28 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:28.833] Received GetPayloadV3                    payloadId=7246
Jul 02 18:54:28 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:28.836] stage running - force txs, with params   txs=9 notxpool=true
Jul 02 18:54:28 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:28.836] [2/7 MiningCreateBlock] Start mine       block=122139213 baseFee=60004124 gasLimit=30000000
Jul 02 18:54:28 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:28.936] Built block                              hash=0x5a45dd65e6341f54f1484cabc72a57618824df9d52cb809824fe5342f3865ca8 height=122139213 txs=9 gas used %=7.030 time=103.283614ms
Jul 02 18:54:28 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:28.940] [NewPayload] Handling new payload        height=122139213 hash=0x5a45dd65e6341f54f1484cabc72a57618824df9d52cb809824fe5342f3865ca8
Jul 02 18:54:28 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:28.974] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
Jul 02 18:54:29 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:29.471] RPC Daemon notified of new headers       from=122139212 to=122139213 amount=1 hash=0x5a45dd65e6341f54f1484cabc72a57618824df9d52cb809824fe5342f3865ca8 header sending=86.981µs log sending=380ns
Jul 02 18:54:29 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:29.472] head updated                             hash=0x5a45dd65e6341f54f1484cabc72a57618824df9d52cb809824fe5342f3865ca8 number=122139213
Jul 02 18:54:29 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:29.483] [ForkChoiceUpdated] BlockBuilder added   payload=7247
Jul 02 18:54:29 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:29.483] Building block...
Jul 02 18:54:29 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:29.484] Received GetPayloadV3                    payloadId=7247
Jul 02 18:54:29 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:29.487] stage running - force txs, with params   txs=15 notxpool=true
Jul 02 18:54:29 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:29.487] [2/7 MiningCreateBlock] Start mine       block=122139214 baseFee=59865354 gasLimit=30000000
Jul 02 18:54:30 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:30.498] Built block                              hash=0xc28882b34616b0bd894dc079dd58b9db8ac18a448939fc99090db63fd270069a height=122139214 txs=15 gas used %=98.442 time=1.01454756s
Jul 02 18:54:30 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:30.508] [NewPayload] Handling new payload        height=122139214 hash=0xc28882b34616b0bd894dc079dd58b9db8ac18a448939fc99090db63fd270069a
Jul 02 18:54:30 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:30.927] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)
Jul 02 18:54:32 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:32.437] RPC Daemon notified of new headers       from=122139213 to=122139214 amount=1 hash=0xc28882b34616b0bd894dc079dd58b9db8ac18a448939fc99090db63fd270069a header sending=24.83µs log sending=470ns
Jul 02 18:54:32 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:32.437] head updated                             hash=0xc28882b34616b0bd894dc079dd58b9db8ac18a448939fc99090db63fd270069a number=122139214
Jul 02 18:54:32 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:32.447] [ForkChoiceUpdated] BlockBuilder added   payload=7248
Jul 02 18:54:32 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:32.447] Building block...
Jul 02 18:54:32 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:32.447] Received GetPayloadV3                    payloadId=7248
Jul 02 18:54:32 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:32.450] stage running - force txs, with params   txs=7 notxpool=true
Jul 02 18:54:32 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:32.450] [2/7 MiningCreateBlock] Start mine       block=122139215 baseFee=61040282 gasLimit=30000000
Jul 02 18:54:32 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:32.659] Built block                              hash=0x10c78a17e2225ab2c9be9d6fc5a78f2613defbd2fb7dac6ea9c59e56c5d2efa8 height=122139215 txs=7 gas used %=5.425 time=212.024533ms
Jul 02 18:54:32 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:32.664] [NewPayload] Handling new payload        height=122139215 hash=0x10c78a17e2225ab2c9be9d6fc5a78f2613defbd2fb7dac6ea9c59e56c5d2efa8
Jul 02 18:54:32 m-optimism-03 erigon[315234]: [INFO] [07-02|18:54:32.694] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload)

op-node:

Jul 02 18:54:02 m-optimism-03 op-node[315258]: t=2024-07-02T18:54:02+0000 lvl=info msg="generated attributes in payload queue" txs=9 timestamp=1719877145
Jul 02 18:54:02 m-optimism-03 op-node[315258]: t=2024-07-02T18:54:02+0000 lvl=info msg="Optimistically queueing unsafe L2 execution payload" id=0x51cd8e3c66365e0685e0d1a63c3ebbb487e1b82abd50f1b1e46770970fcb917e:122173832
Jul 02 18:54:03 m-optimism-03 op-node[315258]: t=2024-07-02T18:54:03+0000 lvl=info msg="inserted block" hash=0x3b4ec4e164333d151673b1dda31fbc70f6a0cde44753f342b1893a4593b2d849 number=122139184 state_root=0xa998cdfb6d12539523c7b3dbb697f90a794fa2eaa5d83216a80ed689e36d9864 timestamp=1719877145 parent=0x860cc01f42cac3e928708b1eb27f340c15e4cfe2367df5349811f1e2dd9a25b7 prev_randao=0x28bce79408966c315f5eb38f659a197e6f672974272ed337eba2e56053ab6d53 fee_recipient=0x4200000000000000000000000000000000000011 txs=9 update_safe=false
Jul 02 18:54:03 m-optimism-03 op-node[315258]: t=2024-07-02T18:54:03+0000 lvl=info msg="generated attributes in payload queue" txs=5 timestamp=1719877147
Jul 02 18:54:03 m-optimism-03 op-node[315258]: t=2024-07-02T18:54:03+0000 lvl=info msg="Received signed execution payload from p2p" id=0x9d87f8180a2d351a4546db984f697c30178e7c2f5e914c6db1ff094e36c9e256:122173833 peer=16Uiu2HAm7dhyfPKohuLyiZ2nUopuNUUdZhJSwxHymwbrGkuuJdZ7
Jul 02 18:54:03 m-optimism-03 op-node[315258]: t=2024-07-02T18:54:03+0000 lvl=info msg="inserted block" hash=0xd5feeb2cf6543858efa4396f3d45de6b800ba96b94326cb76c628c511c305dc7 number=122139185 state_root=0x10efc3cd623364e9f7c45f7c6a48da32d2d96cd472682826378858527ad17874 timestamp=1719877147 parent=0x3b4ec4e164333d151673b1dda31fbc70f6a0cde44753f342b1893a4593b2d849 prev_randao=0x28bce79408966c315f5eb38f659a197e6f672974272ed337eba2e56053ab6d53 fee_recipient=0x4200000000000000000000000000000000000011 txs=5 update_safe=false
Jul 02 18:54:03 m-optimism-03 op-node[315258]: t=2024-07-02T18:54:03+0000 lvl=info msg="generated attributes in payload queue" txs=5 timestamp=1719877149
Jul 02 18:54:03 m-optimism-03 op-node[315258]: t=2024-07-02T18:54:03+0000 lvl=info msg="Optimistically queueing unsafe L2 execution payload" id=0x9d87f8180a2d351a4546db984f697c30178e7c2f5e914c6db1ff094e36c9e256:122173833
Jul 02 18:54:04 m-optimism-03 op-node[315258]: t=2024-07-02T18:54:04+0000 lvl=info msg="inserted block" hash=0xa3453cf2f9d19764e060c572460a1085f7bb1d9d7d15b0814f69da92af23bf40 number=122139186 state_root=0x5edce1374d2d6b9e12a5ff6826e80864ffde268c2cfde7bd20d380c106b61cc6 timestamp=1719877149 parent=0xd5feeb2cf6543858efa4396f3d45de6b800ba96b94326cb76c628c511c305dc7 prev_randao=0x28bce79408966c315f5eb38f659a197e6f672974272ed337eba2e56053ab6d53 fee_recipient=0x4200000000000000000000000000000000000011 txs=5 update_safe=false
Jul 02 18:54:04 m-optimism-03 op-node[315258]: t=2024-07-02T18:54:04+0000 lvl=info msg="generated attributes in payload queue" txs=8 timestamp=1719877151
Jul 02 18:54:05 m-optimism-03 op-node[315258]: t=2024-07-02T18:54:05+0000 lvl=info msg="Received signed execution payload from p2p" id=0x74344fb69b720057a7fd5c1cade782c3e5633360b94fc51cfc406f31560edc60:122173834 peer=16Uiu2HAm7dhyfPKohuLyiZ2nUopuNUUdZhJSwxHymwbrGkuuJdZ7
Jul 02 18:54:05 m-optimism-03 op-node[315258]: t=2024-07-02T18:54:05+0000 lvl=info msg="inserted block" hash=0x81c577cd50bf6a17788d92ea8d75e867c9c95493a3773d91ae96decee7ef1cc2 number=122139187 state_root=0x90a31d75025d9af7341e4d3ad8c7a5392a6d4b58fb416b8ce31c67e8a03ecbcc timestamp=1719877151 parent=0xa3453cf2f9d19764e060c572460a1085f7bb1d9d7d15b0814f69da92af23bf40 prev_randao=0x28bce79408966c315f5eb38f659a197e6f672974272ed337eba2e56053ab6d53 fee_recipient=0x4200000000000000000000000000000000000011 txs=8 update_safe=false
Jul 02 18:54:05 m-optimism-03 op-node[315258]: t=2024-07-02T18:54:05+0000 lvl=info msg="generated attributes in payload queue" txs=6 timestamp=1719877153
Jul 02 18:54:06 m-optimism-03 op-node[315258]: t=2024-07-02T18:54:06+0000 lvl=info msg="inserted block" hash=0x45b47c2e8305462a1705f0b2e37ba36af77cec65d52175fc657c8e6bd78c9b33 number=122139188 state_root=0xf13b1356873f3612595b3ccb0157bea8707ee3e3c4c5977a4bea89fe597b8fe1 timestamp=1719877153 parent=0x81c577cd50bf6a17788d92ea8d75e867c9c95493a3773d91ae96decee7ef1cc2 prev_randao=0x28bce79408966c315f5eb38f659a197e6f672974272ed337eba2e56053ab6d53 fee_recipient=0x4200000000000000000000000000000000000011 txs=6 update_safe=false
Jul 02 18:54:06 m-optimism-03 op-node[315258]: t=2024-07-02T18:54:06+0000 lvl=info msg="generated attributes in payload queue" txs=3 timestamp=1719877155
Jul 02 18:54:07 m-optimism-03 op-node[315258]: t=2024-07-02T18:54:07+0000 lvl=info msg="inserted block" hash=0xa054606e6b69e684227bfc1580784c351a99b62d0734b0df51c57295ee786f6f number=122139189 state_root=0xde39dfcc0179809ee9c4d0ebe4a9cd459c37f1008b1c11e1c41d85e8f2ee65c0 timestamp=1719877155 parent=0x45b47c2e8305462a1705f0b2e37ba36af77cec65d52175fc657c8e6bd78c9b33 prev_randao=0x138df92408c66b6b8b493030d6c3c3e294b048edecd31e0842377c0c609822be fee_recipient=0x4200000000000000000000000000000000000011 txs=3 update_safe=false
Jul 02 18:54:07 m-optimism-03 op-node[315258]: t=2024-07-02T18:54:07+0000 lvl=info msg="generated attributes in payload queue" txs=10 timestamp=1719877157
Jul 02 18:54:07 m-optimism-03 op-node[315258]: t=2024-07-02T18:54:07+0000 lvl=info msg="Optimistically queueing unsafe L2 execution payload" id=0x74344fb69b720057a7fd5c1cade782c3e5633360b94fc51cfc406f31560edc60:122173834
Jul 02 18:54:07 m-optimism-03 op-node[315258]: t=2024-07-02T18:54:07+0000 lvl=info msg="Received signed execution payload from p2p" id=0xa8d81fb3a6309863b2f7fc5ab8cb14fdadefc801fb20dace29e2b20515c47946:122173835 peer=16Uiu2HAmDKnDQXmXFp5P6XoQY4dERM92B7By7U1KEBgfAx8GVbWQ
Jul 02 18:54:08 m-optimism-03 op-node[315258]: t=2024-07-02T18:54:08+0000 lvl=info msg="inserted block" hash=0x840d859dd232406d3a80b705689f29cf9e618b63c06b419b1ec154ab65d317d4 number=122139190 state_root=0x00635c39583125429f552980c00832c3d291695f2345af67a5a8a921e383585f timestamp=1719877157 parent=0xa054606e6b69e684227bfc1580784c351a99b62d0734b0df51c57295ee786f6f prev_randao=0x138df92408c66b6b8b493030d6c3c3e294b048edecd31e0842377c0c609822be fee_recipient=0x4200000000000000000000000000000000000011 txs=10 update_safe=false
Jul 02 18:54:08 m-optimism-03 op-node[315258]: t=2024-07-02T18:54:08+0000 lvl=info msg="Optimistically queueing unsafe L2 execution payload" id=0xa8d81fb3a6309863b2f7fc5ab8cb14fdadefc801fb20dace29e2b20515c47946:122173835
Jul 02 18:54:08 m-optimism-03 op-node[315258]: t=2024-07-02T18:54:08+0000 lvl=info msg="generated attributes in payload queue" txs=7 timestamp=1719877159
Jul 02 18:54:09 m-optimism-03 op-node[315258]: t=2024-07-02T18:54:09+0000 lvl=info msg="Received signed execution payload from p2p" id=0xd6097f9fd537ec5a0ed842bc960249f6f78a7bb2a255a6bdb60970c152602e65:122173836 peer=16Uiu2HAkxfLbtojJACcSDKzdG6QHd8q4TG9MtNCwEhqaneGTCvDw
Jul 02 18:54:09 m-optimism-03 op-node[315258]: t=2024-07-02T18:54:09+0000 lvl=info msg="inserted block" hash=0x1a3ba146dc7f6d2f9337450af6626fda487f91153f24c49054b4e15003fbf941 number=122139191 state_root=0xefe03d627da1e4d1faa4888db989ee1aa8078add112973d892b0bf4d6621b2c4 timestamp=1719877159 parent=0x840d859dd232406d3a80b705689f29cf9e618b63c06b419b1ec154ab65d317d4 prev_randao=0x138df92408c66b6b8b493030d6c3c3e294b048edecd31e0842377c0c609822be fee_recipient=0x4200000000000000000000000000000000000011 txs=7 update_safe=false
Jul 02 18:54:09 m-optimism-03 op-node[315258]: t=2024-07-02T18:54:09+0000 lvl=info msg="generated attributes in payload queue" txs=6 timestamp=1719877161
Jul 02 18:54:09 m-optimism-03 op-node[315258]: t=2024-07-02T18:54:09+0000 lvl=info msg="Optimistically queueing unsafe L2 execution payload" id=0xd6097f9fd537ec5a0ed842bc960249f6f78a7bb2a255a6bdb60970c152602e65:122173836
Jul 02 18:54:10 m-optimism-03 op-node[315258]: t=2024-07-02T18:54:10+0000 lvl=info msg="inserted block" hash=0xa04a7db252ea9473e3a3623799e476dd40018489e6e3f139029d3b6cc928be88 number=122139192 state_root=0xe9b4cf29dcb492079683e6b8df72f7fecccafdc3882ccd10e43d3005260df1df timestamp=1719877161 parent=0x1a3ba146dc7f6d2f9337450af6626fda487f91153f24c49054b4e15003fbf941 prev_randao=0x138df92408c66b6b8b493030d6c3c3e294b048edecd31e0842377c0c609822be fee_recipient=0x4200000000000000000000000000000000000011 txs=6 update_safe=false
Jul 02 18:54:10 m-optimism-03 op-node[315258]: t=2024-07-02T18:54:10+0000 lvl=info msg="generated attributes in payload queue" txs=3 timestamp=1719877163
Jul 02 18:54:10 m-optimism-03 op-node[315258]: t=2024-07-02T18:54:10+0000 lvl=info msg="inserted block" hash=0x8715a28a9ed3997c626720db8afe7c6e0282fccabf8c4c08ff5d45482d7409ba number=122139193 state_root=0xa3a6468372b059037b3326c5f2b416eebca114e9309277724ec2af74cc969e08 timestamp=1719877163 parent=0xa04a7db252ea9473e3a3623799e476dd40018489e6e3f139029d3b6cc928be88 prev_randao=0x138df92408c66b6b8b493030d6c3c3e294b048edecd31e0842377c0c609822be fee_recipient=0x4200000000000000000000000000000000000011 txs=3 update_safe=false
andreclaro commented 4 months ago

OP mainnet node has been syncing from the latest snapshot since yesterday afternoon (UTC timezone) and it is still 16 hours behind and not catching up. Block height every 60 seconds:

[2024-07-03 08:02:24] Latest Block Number: 122167512
[2024-07-03 08:03:24] Latest Block Number: 122167565
[2024-07-03 08:04:24] Latest Block Number: 122167610
[2024-07-03 08:05:24] Latest Block Number: 122167657
[2024-07-03 08:06:24] Latest Block Number: 122167701
[2024-07-03 08:07:24] Latest Block Number: 122167751
[2024-07-03 08:08:24] Latest Block Number: 122167797
[2024-07-03 08:09:24] Latest Block Number: 122167843
[2024-07-03 08:10:24] Latest Block Number: 122167893
[2024-07-03 08:11:24] Latest Block Number: 122167938
[2024-07-03 08:12:24] Latest Block Number: 122167988
[2024-07-03 08:13:24] Latest Block Number: 122168036
[2024-07-03 08:14:24] Latest Block Number: 122168086
mininny commented 4 months ago

@andreclaro that's weird. I'll launch up a new ec2 instance with the configurations same as yours, and see how the node syncs. I'll update you on the results afterwards.

andreclaro commented 4 months ago

it is now 8 hrs behind... on block 122197699: https://optimistic.etherscan.io/block/122197699

I will keep the node running for more a few days... what ebs volume do you use? gp3 or io2? and iops?

Thanks

andreclaro commented 4 months ago

@mininny , The node is now in sync. We used io2 ebs volume with 30k iops, however I would like to know if gp3 with 16k iops is enough?!?!? Thanks

mininny commented 4 months ago

@mininny , The node is now in sync. We used io2 ebs volume with 30k iops, however I would like to know if gp3 with 16k iops is enough?!?!? Thanks

@andreclaro That's great to hear. :)

Erigon does require low disk latency, and gp3 isn't fast enough which is why it struggles in the initial sync phase. Once it catches up to the head, it might be able to sync, but I think it would be better to use io-type ebs volume or something like i3 ec2 instances if you want fast syncing.

You can also find related discussion about disk types and required iops in the upstream erigon issues: https://github.com/ledgerwatch/erigon