L2OO falls behind L2 in devnet

karlb commented 2 months ago

It looks like https://github.com/ethereum-optimism/optimism/issues/7668 is still a problem, although to a lesser degree, since l2OutputOracleSubmissionInterval has been increased since that issue has been originally reported. I can't reopen that issue or comment on it, so I had to create a new one.

https://github.com/ethereum-optimism/optimism/issues/7668#issuecomment-2174503801 says that his should not happen when fault proofs are enabled, but my observation seems to disagree.

Steps to reproduce

Just run a devnet and watch how the block number in the L2OO falls behind over time.

git checkout b6e9a614fb4c3d0f0089d0353f9d3ccc7c0be3df
DEVNET_L2OO=true make devnet-clean devnet-up;
while true
    set L1 $(cast block -r http://localhost:8545 -f number)
    set L2 $(cast block -r http://localhost:9545 -f number)
    set L2OO $(cast call -r http://localhost:8545 $(jq '.L2OutputOracleProxy' ~/optimism/.devnet/addresses.json -r) 'latestBlockNumber() returns (uint256)')
    echo L1: $L1, L2: $L2, L2OO: $L2OO, L2OO-diff: $(math "$L2 - $L2OO")
    sleep 30
end

Expected output

I would expect the L2OO-diff (the difference between the latest block number in the L2OO and the latest block number on the L2) to be constant over time (minus some noise).

Actual output

L2OO-diff increases over time:

L1: 1, L2: 0, L2OO: 0, L2OO-diff: 0
L1: 6, L2: 16, L2OO: 0, L2OO-diff: 16
L1: 11, L2: 31, L2OO: 10, L2OO-diff: 21
L1: 16, L2: 46, L2OO: 20, L2OO-diff: 26
L1: 21, L2: 61, L2OO: 30, L2OO-diff: 31
L1: 26, L2: 77, L2OO: 40, L2OO-diff: 37
L1: 31, L2: 92, L2OO: 60, L2OO-diff: 32
L1: 36, L2: 107, L2OO: 70, L2OO-diff: 37
L1: 41, L2: 122, L2OO: 80, L2OO-diff: 42
L1: 46, L2: 137, L2OO: 90, L2OO-diff: 47
L1: 51, L2: 152, L2OO: 110, L2OO-diff: 42
L1: 56, L2: 167, L2OO: 120, L2OO-diff: 47
L1: 61, L2: 182, L2OO: 130, L2OO-diff: 52
L1: 66, L2: 197, L2OO: 140, L2OO-diff: 57
L1: 71, L2: 213, L2OO: 160, L2OO-diff: 53
L1: 77, L2: 228, L2OO: 170, L2OO-diff: 58
L1: 82, L2: 243, L2OO: 180, L2OO-diff: 63
L1: 87, L2: 258, L2OO: 200, L2OO-diff: 58
L1: 92, L2: 273, L2OO: 210, L2OO-diff: 63
L1: 97, L2: 288, L2OO: 220, L2OO-diff: 68
L1: 102, L2: 303, L2OO: 230, L2OO-diff: 73
L1: 107, L2: 318, L2OO: 250, L2OO-diff: 68
L1: 112, L2: 333, L2OO: 260, L2OO-diff: 73
L1: 117, L2: 348, L2OO: 270, L2OO-diff: 78
L1: 122, L2: 363, L2OO: 280, L2OO-diff: 83
L1: 127, L2: 379, L2OO: 300, L2OO-diff: 79
L1: 132, L2: 394, L2OO: 310, L2OO-diff: 84
L1: 137, L2: 409, L2OO: 320, L2OO-diff: 89
L1: 142, L2: 424, L2OO: 330, L2OO-diff: 94
L1: 147, L2: 439, L2OO: 350, L2OO-diff: 89
L1: 152, L2: 454, L2OO: 360, L2OO-diff: 94
L1: 157, L2: 469, L2OO: 370, L2OO-diff: 99
L1: 162, L2: 484, L2OO: 380, L2OO-diff: 104
L1: 167, L2: 499, L2OO: 400, L2OO-diff: 99
L1: 172, L2: 515, L2OO: 410, L2OO-diff: 105

Workaround

Increasing l2OutputOracleSubmissionInterval to 15 solves the issue and leads to the following output:

L1: 1, L2: 0, L2OO: 0, L2OO-diff: 0
L1: 6, L2: 16, L2OO: 0, L2OO-diff: 16
L1: 11, L2: 31, L2OO: 0, L2OO-diff: 31
L1: 16, L2: 46, L2OO: 15, L2OO-diff: 31
L1: 21, L2: 61, L2OO: 30, L2OO-diff: 31
L1: 26, L2: 76, L2OO: 45, L2OO-diff: 31
L1: 31, L2: 92, L2OO: 60, L2OO-diff: 32
L1: 36, L2: 107, L2OO: 75, L2OO-diff: 32
L1: 41, L2: 122, L2OO: 90, L2OO-diff: 32
L1: 46, L2: 137, L2OO: 105, L2OO-diff: 32
L1: 51, L2: 152, L2OO: 120, L2OO-diff: 32
L1: 56, L2: 167, L2OO: 135, L2OO-diff: 32
L1: 61, L2: 182, L2OO: 150, L2OO-diff: 32
L1: 66, L2: 197, L2OO: 165, L2OO-diff: 32
L1: 71, L2: 212, L2OO: 180, L2OO-diff: 32
L1: 77, L2: 228, L2OO: 210, L2OO-diff: 18
L1: 82, L2: 243, L2OO: 210, L2OO-diff: 33
L1: 87, L2: 258, L2OO: 240, L2OO-diff: 18
L1: 92, L2: 273, L2OO: 240, L2OO-diff: 33
L1: 97, L2: 288, L2OO: 270, L2OO-diff: 18
L1: 102, L2: 303, L2OO: 270, L2OO-diff: 33
L1: 107, L2: 318, L2OO: 300, L2OO-diff: 18
L1: 112, L2: 333, L2OO: 300, L2OO-diff: 33
L1: 117, L2: 348, L2OO: 330, L2OO-diff: 18
L1: 122, L2: 364, L2OO: 330, L2OO-diff: 34
L1: 127, L2: 379, L2OO: 360, L2OO-diff: 19
L1: 132, L2: 394, L2OO: 360, L2OO-diff: 34
L1: 137, L2: 409, L2OO: 390, L2OO-diff: 19

tynes commented 2 months ago

When fault proofs are enabled, the L2OutputOracle is no longer in the hot path of the system. I recommend using the fault proof based system with the permissioned dispute game if you plan on launching a new chain that doesn't have a live proof system. This would be very similar to how the L2OutputOracle works in practice.

The real fix for this problem would be to make the proposer send transactions in parallel. It falls behind due to it waiting for the transaction to be included before sending the next one.

karlb commented 1 month ago

OK, makes sense. Should we close this as "won't fix" then, or is anyone interested in improving the situation for the L2OO?

ethereum-optimism / optimism