reth on base mainnet not working with op-node v1.5.0

[Chomtana]

Describe the bug

Base mainnet op-reth stuck with "no parent beacon block root post-cancun" when using with op-node v1.5.0

Also, op-node is stuck with many "no peers ready to handle block requests for more P2P requests for L2 block history"

Please see the longer log at https://gist.github.com/Chomtana/ff1b6181e6b0f424ac1e0111c89696ff because GitHub says it's too long

Notice that op-node v1.5.0 uses engine_forkchoiceUpdatedV3 while v1.4.3 uses engine_forkchoiceUpdatedV2

Steps to reproduce

1. Install and run the latest op-reth or v0.1.0-alpha.16
2. Install and run op-node v1.5.0
3. It stuck...

To fix this, I revert op-node to v1.4.3 and op-reth to v0.1.0-alpha.16 (Release)

Node logs

Feb 04 04:37:20 node op-reth[505863]: 2024-02-04T04:37:20.723158Z  INFO blockchain_tree: Block is already canonical, ignoring. block_hash=0x842eb0ba12975b0243c977e6644e4ff043d23a518055f1d                                                                                                                               e3515e8315a67c6a9
Feb 04 04:37:20 node op-reth[505863]: 2024-02-04T04:37:20.723343Z  INFO reth::commands::node::events: Canonical chain committed number=10094800 hash=0x842eb0ba12975b0243c977e6644e4ff043d2                                                                                                                               3a518055f1de3515e8315a67c6a9 elapsed=212.838µs
Feb 04 04:37:20 node op-reth[505863]: 2024-02-04T04:37:20.723408Z  INFO reth::commands::node::events: Forkchoice updated head_block_hash=0x842eb0ba12975b0243c977e6644e4ff043d23a518055f1de                                                                                                                               3515e8315a67c6a9 safe_block_hash=0x842eb0ba12975b0243c977e6644e4ff043d23a518055f1de3515e8315a67c6a9 finalized_block_hash=0x842eb0ba12975b0243c977e6644e4ff043d23a518055f1de3515e8315a67c6a9 st                                                                                                                               atus=Valid
Feb 04 04:37:20 node op-reth[505863]: 2024-02-04T04:37:20.723418Z TRACE method_call{method="engine_forkchoiceUpdatedV3"}: jsonrpsee_core::tracing: send="{\"jsonrpc\":\"2.0\",\"error\":{\"                                                                                                                               code\":-32602,\"message\":\"Invalid params\",\"data\":{\"err\":\"no parent beacon block root post-cancun\"}},\"id\":1896}"
Feb 04 04:37:22 node op-reth[505863]: 2024-02-04T04:37:22.435541Z  INFO reth::cli: Status connected_peers=0 freelist=54190 latest_block=10094800

Platform(s)

Linux (x86)

What version/commit are you on?

commit 10f4434

What database version are you on?

Current database version: 1 Local database version: 1

What type of node are you running?

Archive (default)

What prune config do you use, if any?

No response

If you've built Reth from source, provide the full command you used

make install-op

Code of Conduct

• [X] I agree to follow the Code of Conduct

[mattsse]

@clabby does op-node support the

parentBeaconBlockRoot: DATA, 32 Bytes - Root of the parent beacon block.

https://github.com/ethereum/execution-apis/blob/main/src/engine/cancun.md#payloadattributesv3

looks like it does

https://github.com/ethereum-optimism/optimism/blob/dcdf044f3a5ce39afdf67e9efbb8ec3f1b9b0be4/op-service/eth/types.go#L302

[clabby]

@mattsse Yes, it does after Dencun activation (in the Ecotone hardfork, which has not been activated anywhere except for internal devnets just yet). https://specs.optimism.io/protocol/exec-engine.html#extended-payloadattributesv3

Starting at Ecotone, the parentBeaconBlockRoot must be set to the L1 origin parentBeaconBlockRoot, or a zero bytes32 if the Dencun functionality with parentBeaconBlockRoot is not active on L1.

[Rjected]

@mattsse Yes, it does after Dencun activation (in the Ecotone hardfork, which has not been activated anywhere except for internal devnets just yet). https://specs.optimism.io/protocol/exec-engine.html#extended-payloadattributesv3

Starting at Ecotone, the parentBeaconBlockRoot must be set to the L1 origin parentBeaconBlockRoot, or a zero bytes32 if the Dencun functionality with parentBeaconBlockRoot is not active on L1.

@clabby Do you know why the above logs say that the engine_forkchoiceUpdatedV3 method was called then? V3 methods should only be called post-ecotone IIRC

[clabby]

@Rjected This is on our end, got to the bottom of it a few mins ago. Fix in review: https://github.com/ethereum-optimism/optimism/pull/9253

geth has identical behavior to reth in upstream (where it rejects V3 messages w/ a missing parentBeaconBlockRoot), merged recently w/ https://github.com/ethereum/go-ethereum/pull/28230. We've pulled in the upstream PR, and an rc for v1.5.1 will be released once 9253 is merged.

@Chomtana For now, temporarily downgrading to v1.4.3 will do the trick.

[Rjected]

@Rjected This is on our end, got to the bottom of it a few mins ago. Fix in review: ethereum-optimism/optimism#9253

geth has identical behavior to reth in upstream (where it rejects V3 messages w/ a missing parentBeaconBlockRoot), merged recently w/ ethereum/go-ethereum#28230. We've pulled in the upstream PR, and an rc for v1.5.1 will be released once 9253 is merged.

@Chomtana For now, temporarily downgrading to v1.4.3 will do the trick.

awesome, that makes sense. I'll watch that PR, when it's merged + released, I'll close this issue with a recommendation to update op-node

[Rjected]

The fix is now merged and available in a release candidate: https://github.com/ethereum-optimism/optimism/releases/tag/op-node%2Fv1.5.1-rc.1

closing this!