search

ethereum-optimism / developers

This repository is to serve as a place where builders on every level of the OP Stack can come to collaborate.
Creative Commons Zero v1.0 Universal
71 stars 44 forks source link

Second node created a fork and stuck #572

Closed askucher closed 3 weeks ago

askucher commented 3 weeks ago

Bug Description We have launched version 1.9.0 of Optimism along with beta.2 of our smart contracts using the useCustomToken feature. Everything is functioning correctly on our servers, and blocks are being created as expected.

To enhance fault tolerance, we decided to deploy additional nodes, using the example from this guide.

However, after launching a second node (op-geth + op-node), it started syncing but eventually created a fork and got stuck. This was unexpected because the setup should have maintained synchronization across nodes without forking

Thats why we have some questions

  1. How the optimism is protected from L1 forks as far we use some L1 rpc which can be unstable
  2. Can we make so optimism can roll back some block and try to resolve itself automatically
  3. How is fork possible in case when we have 1 block proposer?

Steps to Reproduce

  1. There is not steps to reproduce but we have some questions

Expected behavior It's should working perfectly without forks and we need to have possibility to switch from one node to another is something happened with a disk

Environment Information:

Configurations: We do not use the super chain registry all vars are following the documentation.

Logs: 7|op-node | t=2024-09-02T15:11:11+0000 lvl=info msg="Scheduled sequencer action" delta=1.999644808s 7|op-node | t=2024-09-02T15:11:12+0000 lvl=warn msg="unexpected block author" topic=blocksV3 err= peer=16Uiu2HAmLPbSkxyYGYiqaBHyCP1SdFwpVcLhqtTDnj64gEzq1Kmn addr=0x99d508fAd3d9aeE9eA4f206191c6dB915221cE12 expected=0x7b04384c0a67E27E95c508c8B8D79afA9080f5f5

Info about first block where is different state 

Detailed INFO OF BLOCKS 217 - REMOTE

baseFeePerGas        422404275
difficulty           0
extraData            0x
gasLimit             30000000
gasUsed              43890
hash                 0xb54822428dbd243d1d61c4460c84a18b990178f6834e4092ba0b2df4e1831361
logsBloom            0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
miner                0x4200000000000000000000000000000000000011
mixHash              0xb909be732042d9972431e777d8e7c0468f2654961cb88a6330b993c5d73b03ae
nonce                0x0000000000000000
number               217
parentHash           0x9725e439d7eec1b659ffc22548b81597a33c3344f1c133f1dfb50bd92aa72bf3
transactionsRoot     0x81863caf566c2af30e72fe923297fb7a9cc7eb29136ab8d70e84c5f415d9616b
receiptsRoot         0x018603f8ce61ad4aa03e27b84fe3f996320b7a275d3cb996add4e628c7602a8f
sha3Uncles           0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347
size                 840
stateRoot            0x64ff565bfa20da241af389acd56c0c741077419354e89a5d9263616b34138d1f
timestamp            1724783006 (Tue, 27 Aug 2024 18:23:26 +0000)
withdrawalsRoot      0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421
totalDifficulty      0
transactions:        [
    0xb8c012f2202e66e31d82367446fb89435d543037c8ee92e2dca9dd2f03800c9b
]
Detailed INFO OF BLOCKS 217 - LOCAL

baseFeePerGas        422404275
difficulty           0
extraData            0x
gasLimit             30000000
gasUsed              55066
hash                 0x07552e9c0789c60951d3a229bbdad911de69a5912cac713939d8b2d860b6f244
logsBloom            0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
miner                0x4200000000000000000000000000000000000011
mixHash              0xd8404809117914e393a11a167c873e2e1145931cd311e08d3255e06704f82804
nonce                0x0000000000000000
number               217
parentHash           0x9725e439d7eec1b659ffc22548b81597a33c3344f1c133f1dfb50bd92aa72bf3
transactionsRoot     0xbc4b028985f76b1f5941c90dc7e9b6785d82b7a21d5c24681659a38e56d54d3b
receiptsRoot         0x8435f38e8666eb2f15cd135f9f46fedeed9265ae5b399d1f80c7b19c69d12909
sha3Uncles           0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347
size                 840
stateRoot            0x6757ae4ec81e7adcd5f8cfafea686aa0da3fdcb8a3be0d7c9fbff0bc4c792d6d
timestamp            1724783006 (Tue, 27 Aug 2024 18:23:26 +0000)
withdrawalsRoot      0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421
totalDifficulty      0
transactions:        [
    0xb7c4248afcf325c1339b370cafe2957d42a976814db17525c6984cb533d8ae40
]
Detailed INFO OF TX 0xb8c012f2202e66e31d82367446fb89435d543037c8ee92e2dca9dd2f03800c9b - REMOTE

blockHash            0xb54822428dbd243d1d61c4460c84a18b990178f6834e4092ba0b2df4e1831361
blockNumber          217
from                 0xDeaDDEaDDeAdDeAdDEAdDEaddeAddEAdDEAd0001
gas                  1000000
gasPrice             0
hash                 0xb8c012f2202e66e31d82367446fb89435d543037c8ee92e2dca9dd2f03800c9b
input                0x440a5e20000f42400000000000000000000000040000000066ce1954000000000064720e000000000000000000000000000000000000000000000000000000040cf4e7e40000000000000000000000000000000000000000000000000000006304a6b4ac86ae18a7696c27438365d637551fffd1c5ea6319e2be61ffd7b76d655699a184000000000000000000000000f36af909d472d966c49eb11217efc90fd070abd4
nonce                216
r                    0x0000000000000000000000000000000000000000000000000000000000000000
s                    0x0000000000000000000000000000000000000000000000000000000000000000
to                   0x4200000000000000000000000000000000000015
transactionIndex     0
v                    0
value                0
depositReceiptVersion 1
mint                 0
sourceHash           39140696042246058621864761218877383772630202131977715764031800604037221138676

Detailed INFO OF TX 0xb7c4248afcf325c1339b370cafe2957d42a976814db17525c6984cb533d8ae40 - REMOTE

blockHash            0x3a1ab5ca2462ce034636a95f729af89267c7419fc8b4c4a6566d1ac6320ae1bd
blockNumber          218
from                 0xDeaDDEaDDeAdDeAdDEAdDEaddeAddEAdDEAd0001
gas                  1000000
gasPrice             0
hash                 0xb7c4248afcf325c1339b370cafe2957d42a976814db17525c6984cb533d8ae40
input                0x440a5e20000f42400000000000000000000000000000000066ce1960000000000064720f00000000000000000000000000000000000000000000000000000004033f17100000000000000000000000000000000000000000000000000000006f653ba46682400829565b5287dfd4004df64e029b876b7532e1ca98947114e0c347b10367000000000000000000000000f36af909d472d966c49eb11217efc90fd070abd4
nonce                217
r                    0x0000000000000000000000000000000000000000000000000000000000000000
s                    0x0000000000000000000000000000000000000000000000000000000000000000
to                   0x4200000000000000000000000000000000000015
transactionIndex     0
v                    0
value                0
depositReceiptVersion 1
mint                 0
sourceHash           17760142539965865497085337990672279271964787131804171250706484526030666864411

Detailed INFO OF TX 0xb7c4248afcf325c1339b370cafe2957d42a976814db17525c6984cb533d8ae40 - LOCAL

blockHash            0x07552e9c0789c60951d3a229bbdad911de69a5912cac713939d8b2d860b6f244
blockNumber          217
from                 0xDeaDDEaDDeAdDeAdDEAdDEaddeAddEAdDEAd0001
gas                  1000000
gasPrice             0
hash                 0xb7c4248afcf325c1339b370cafe2957d42a976814db17525c6984cb533d8ae40
input                0x440a5e20000f42400000000000000000000000000000000066ce1960000000000064720f00000000000000000000000000000000000000000000000000000004033f17100000000000000000000000000000000000000000000000000000006f653ba46682400829565b5287dfd4004df64e029b876b7532e1ca98947114e0c347b10367000000000000000000000000f36af909d472d966c49eb11217efc90fd070abd4
nonce                216
r                    0x0000000000000000000000000000000000000000000000000000000000000000
s                    0x0000000000000000000000000000000000000000000000000000000000000000
to                   0x4200000000000000000000000000000000000015
transactionIndex     0
v                    0
value                0
depositReceiptVersion 1
mint                 0
sourceHash           17760142539965865497085337990672279271964787131804171250706484526030666864411

Detailed INFO OF TX 0xb8c012f2202e66e31d82367446fb89435d543037c8ee92e2dca9dd2f03800c9b - LOCAL
Error: 
tx not found: 0xb8c012f2202e66e31d82367446fb89435d543037c8ee92e2dca9dd2f03800c9b

Additional context We can provide more information per request

⚠️ Notice: Issues that do not include the following sections will be subject to closure:

Please ensure all required sections are filled out accurately to expedite the debugging process and improve issue resolution efficiency.