trace_block: unexpected empty response on empty blocks post Merge

[ncocchiaro]

System information

Erigon version: ./erigon --version: erigon version 2.60.1

OS & Version: Windows/Linux/OSX: Linux Ubuntu 22.04

Commit hash: 9471c44e04f43a735dc9f40b74f552d706d05f78

Erigon Command (with flags/config):

/usr/bin/erigon --db.size.limit=8TB --db.pagesize=4KB --chain=sepolia  --pprof --rpc.gascap 500000000 --nat=stun  --authrpc.addr=0.0.0.0 --authrpc.vhosts=* --http --http.addr 0.0.0.0 --http.corsdomain * --http.port 8545 --http.api eth,trace,net,engine,web3 --http.vhosts * --port 30303 --maxpeers 50 --txpool.nolocals --txpool.accountslots 32 --txpool.globalslots 8192 --txpool.accountqueue 128 --txpool.globalqueue 2048 --txpool.lifetime 3h0m0s --datadir /mnt/local/ethereum --ws    --authrpc.jwtsecret /mnt/local/ethereum/engine-jwt.key

Consensus Layer: teku/v24.2.0/linux-x86_64/-ubuntu-openjdk64bitservervm-java-17

Consensus Layer Command (with flags/config):

/teku --log-destination=console --data-base-path=/mnt/local/ethereum --network=sepolia --ee-endpoint=http://localhost:8551 --eth1-endpoint=http://localhost:8545 --ee-jwt-secret-file=/mnt/local/ethereum/engine-jwt.key --rest-api-enabled=true --rest-api-host-allowlist=* --rest-api-interface=0.0.0.0 --rest-api-port=5051 --metrics-enabled=true --metrics-host-allowlist=* --metrics-interface=0.0.0.0 --metrics-port=8008 --metrics-block-timing-tracking-enabled=true --checkpoint-sync-url=https://beaconstate-sepolia.chainsafe.io

Chain/Network: Ethereum Sepolia

Expected behaviour

Calling trace_block on an empty Sepolia block like 1450408 (0x1621a8) or 1450409 (0x1621a9) should produce a response like the following:

$ curl -X "POST" "http://localhost:8545" -H 'Content-Type: application/json' -d $'{"id": 1,"method": "trace_block","jsonrpc": "2.0","params": ["0x1621a8"]}'

{
  "jsonrpc": "2.0",
  "id": 1,
  "result": [
    {
      "action": {
        "author": "0xedf0f83b6da795931f668e64d82eb8c5aad46340",
        "rewardType": "block",
        "value": "0x1bc16d674ec80000"
      },
      "blockHash": "0xd07cce9785d39c0dd2409b7d8e69d6bff26a69a0fa5308ac781c63ffe2a37bc1",
      "blockNumber": 1450408,
      "result": null,
      "subtraces": 0,
      "traceAddress": [],
      "type": "reward"
    }
  ]
}

We would expect the two empty blocks to produce similar responses.

Actual behaviour

The call from the example above produces the expected response when made against Sepolia block 1450408 (0x1621a8). However, when the call is made against block 1450409 (0x1621a9), the result field is entirely empty:

$ curl -X "POST" "http://localhost:8545" -H 'Content-Type: application/json' -d $'{"id": 1,"method": "trace_block","jsonrpc": "2.0","params": ["0x1621a9"]}

{
  "jsonrpc": "2.0",
  "id": 1,
  "result": []
}

Steps to reproduce the behaviour

Run the listed versions of erigon and teku to sync Ethereum Sepolia from scratch, then make the trace_block calls as shown via curl.

[ncocchiaro]

Updated the title of the issue. I was able to verify that this also happens on Ethereum Mainnet, with empty blocks after the Merge block (0xed14f1), for example 0xed175b.

On Sepolia, the blocks we were testing with happened to be on either side of the Merge, so for 0x1621a8 (last block before the Merge) we get a full-fledged response, whereas for 0x1621a9 (first Merge block) we get the empty result.

[ncocchiaro]

This behavior also looks to be present in erigon 2.58.1.