search

kroma-network / go-ethereum

go-ethereum for Kroma
GNU Lesser General Public License v3.0
44 stars 23 forks source link

Persistent Sync Deviation in op-geth #85

Closed andrewten123 closed 5 months ago

andrewten123 commented 6 months ago

Bug Description After updating to version v1.101304.1, op-geth is consistently lagging behind the actual chain height by approximately 20,000 blocks. We have performed a full node resync several times, but it hasn't yielded any results; the lag persists.

Steps to Reproduce run the node

Expected behavior op-geth should sync with the latest block height in the chain.

Environment Information: kro-geth version: v0.4.3 kro-node version: v1.3.2

Logs: ``` INFO [03-17|12:44:24.654] --------------------------------------------------------------------------------------------------------------------------------------------------------- INFO [03-17|12:44:24.654] Chain ID: 255 (KromaMainnet) INFO [03-17|12:44:24.654] Consensus: Kroma INFO [03-17|12:44:24.654] INFO [03-17|12:44:24.654] Pre-Merge hard forks (block based): INFO [03-17|12:44:24.654] - Homestead: #0 (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/homestead.md) INFO [03-17|12:44:24.654] - Tangerine Whistle (EIP 150): #0 (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/tangerine-whistle.md) INFO [03-17|12:44:24.654] - Spurious Dragon/1 (EIP 155): #0 (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/spurious-dragon.md) INFO [03-17|12:44:24.654] - Spurious Dragon/2 (EIP 158): #0 (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/spurious-dragon.md) INFO [03-17|12:44:24.654] - Byzantium: #0 (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/byzantium.md) INFO [03-17|12:44:24.654] - Constantinople: #0 (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/constantinople.md) INFO [03-17|12:44:24.654] - Petersburg: #0 (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/petersburg.md) INFO [03-17|12:44:24.654] - Istanbul: #0 (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/istanbul.md) INFO [03-17|12:44:24.654] - Muir Glacier: #0 (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/muir-glacier.md) INFO [03-17|12:44:24.654] - Berlin: #0 (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/berlin.md) INFO [03-17|12:44:24.654] - London: #0 (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/london.md) INFO [03-17|12:44:24.654] - Arrow Glacier: #0 (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/arrow-glacier.md) INFO [03-17|12:44:24.654] - Gray Glacier: #0 (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/gray-glacier.md) INFO [03-17|12:44:24.654] INFO [03-17|12:44:24.654] Merge configured: INFO [03-17|12:44:24.654] - Hard-fork specification: https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/paris.md INFO [03-17|12:44:24.654] - Network known to be merged: true INFO [03-17|12:44:24.654] - Total terminal difficulty: 0 INFO [03-17|12:44:24.654] - Merge netsplit block: #0 INFO [03-17|12:44:24.654] INFO [03-17|12:44:24.654] Post-Merge hard forks (timestamp based): INFO [03-17|12:44:24.654] - Shanghai: @1708502400 (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/shanghai.md) INFO [03-17|12:44:24.654] - Regolith: @0 INFO [03-17|12:44:24.654] - Canyon: @1708502400 INFO [03-17|12:44:24.654] INFO [03-17|12:44:24.654] --------------------------------------------------------------------------------------------------------------------------------------------------------- INFO [03-17|12:44:24.654] INFO [03-17|12:44:24.654] Loaded most recent local block number=8,377,619 hash=fadcc5..314395 td=0 age=12h10m39s INFO [03-17|12:44:24.654] Loaded most recent local finalized block number=8,377,127 hash=70fb9d..deb854 td=0 age=12h27m3s INFO [03-17|12:44:24.655] Initialising Ethereum protocol network=255 dbversion=8 INFO [03-17|12:44:24.655] Initialized transaction indexer limit=0 INFO [03-17|12:44:24.659] Setting new local account address=0xda9e8e71bB750a996Af33ebB8aBb18cd9EB9DC75 INFO [03-17|12:44:24.659] Loaded local transaction journal transactions=2 dropped=0 INFO [03-17|12:44:24.660] Regenerated local transaction journal transactions=2 accounts=1 INFO [03-17|12:44:24.668] Chain post-merge, sync via beacon client INFO [03-17|12:44:24.668] Gasprice oracle is ignoring threshold set threshold=2 WARN [03-17|12:44:24.673] Engine API enabled protocol=eth INFO [03-17|12:44:24.673] Starting peer-to-peer node instance=Geth/v0.1.0-unstable-af3965bb/linux-amd64/go1.21.7 INFO [03-17|12:44:24.737] New local node record seq=1,709,399,426,984 id=61a87a1bcb78ae69 ip=127.0.0.1 udp=30303 tcp=30304 INFO [03-17|12:44:24.738] Started P2P networking self="enode://d1a7e4b8ef76db8b05c176d197d7648e639c7430e6665e0371ce2baa42c2e298d228f95705faadfae217f13e2bb6bf4466a992d9c7924a4a961d6d82bf95221f@127.0.0.1:30304?discport=30303" INFO [03-17|12:44:24.740] IPC endpoint opened url=/.kroma/db/geth.ipc INFO [03-17|12:44:24.740] Loaded JWT secret file path=/local/jwtsecret crc32=0x9c511b3a INFO [03-17|12:44:24.741] HTTP server started endpoint=[::]:8545 auth=false prefix= cors=* vhosts=* INFO [03-17|12:44:24.741] WebSocket enabled url=ws://[::]:8546 INFO [03-17|12:44:24.741] WebSocket enabled url=ws://[::]:8551 INFO [03-17|12:44:24.741] HTTP server started endpoint=[::]:8551 auth=true prefix= cors=localhost vhosts=* ```
0xHansLee commented 6 months ago

It might be forked from the batch. That is, your kroma-node might drop the batch posted to L1 and generated a new batch. Can you see a log like below? "Generating a new batch"

andrewten123 commented 5 months ago

Yes, there is indeed "Generating a new batch" in the logs. Can you advise on how we can fix this?

Logs: ``` t=2024-03-21T11:33:34+0000 lvl=info msg="skipping unsafe payload, since it does not build onto the existing unsafe chain" safe=0x24785e1e401edec1c804f965147776f6259674d7b329683a75a4425135a86ff6:8548358 unsafe=0x704f7e27664bea0516eeece28d748d45b16debfc2aa1dbcdd8fbf51ff3cc70de:8548359 payload=0x704f7e27664bea0516eeece28d748d45b16debfc2aa1dbcdd8fbf51ff3cc70de:8548359 t=2024-03-21T11:33:34+0000 lvl=info msg="Generating next batch" epoch=0x52b9da1dfd6fa0ad078af42212fb99c4a2c418be62730d91107a9a92a2c2ba15:19479166 timestamp=1,710,977,105 t=2024-03-21T11:33:34+0000 lvl=info msg="generated attributes in payload queue" txs=1 timestamp=1,710,977,105 t=2024-03-21T11:33:34+0000 lvl=info msg="inserted block" hash=0x22c698422e1ef77c2b2e44a082cdd67af0de6629d07d6ce2e1eca4da60d42da2 number=8,548,359 state_root=0x1e3e869252805dc6d8a14296f62dd57da0a7cfc31f4782aa7ae623eb8b608499 timestamp=1,710,977,105 parent=0x24785e1e401edec1c804f965147776f6259674d7b329683a75a4425135a86ff6 prev_randao=0xa00780e3acd6ac4426d807bed5c2de9ef83e2759434dee1c4b0474aa3e0acea5 fee_recipient=0x0000000000000000000000000000000000000000 txs=1 update_safe=true t=2024-03-21T11:33:34+0000 lvl=info msg="Sync progress" reason="processed safe block derived from L1" l2_finalized=0xca45d5d92986ebf90206fa23dd1f4a245160e8391be2c97b32512fe5ae4fe230:8547845 l2_safe=0x22c698422e1ef77c2b2e44a082cdd67af0de6629d07d6ce2e1eca4da60d42da2:8548359 l2_safe_pending=0x22c698422e1ef77c2b2e44a082cdd67af0de6629d07d6ce2e1eca4da60d42da2:8548359 l2_unsafe=0x22c698422e1ef77c2b2e44a082cdd67af0de6629d07d6ce2e1eca4da60d42da2:8548359 l2_engineSyncTarget=0x22c698422e1ef77c2b2e44a082cdd67af0de6629d07d6ce2e1eca4da60d42da2:8548359 l2_time=1,710,977,105 l1_derived=0x846be25ca25b34a31d62095739d2fa1344d7f13bc75f285f84f1df2fd070cd31:19482766 t=2024-03-21T11:33:34+0000 lvl=info msg="skipping unsafe payload, since it does not build onto the existing unsafe chain" safe=0x22c698422e1ef77c2b2e44a082cdd67af0de6629d07d6ce2e1eca4da60d42da2:8548359 unsafe=0xfe12afb3e67cd8aebf7c36b1c74af718f1b6e3e61a1e9b2ac0359b4f9c754eca:8548360 payload=0xfe12afb3e67cd8aebf7c36b1c74af718f1b6e3e61a1e9b2ac0359b4f9c754eca:8548360 t=2024-03-21T11:33:34+0000 lvl=info msg="Generating next batch" epoch=0x52b9da1dfd6fa0ad078af42212fb99c4a2c418be62730d91107a9a92a2c2ba15:19479166 timestamp=1,710,977,107 t=2024-03-21T11:33:34+0000 lvl=info msg="generated attributes in payload queue" txs=1 timestamp=1,710,977,107` ```
0xHansLee commented 5 months ago

@andrewten123 If so, your node has been forked. You need to sync the block from the snapshot we provided. Please fetch our snapshot again. When you set up your node, please check the upgrade instruction related to the version. There are some changes in rollup.json and .env file. This is actually not the issue, so I will close this issue. If you have any helps, please visit our discord.