Inconsistent `blockNumber` in Transaction Receipts for Block 2285347 on Moonbase Alpha

[JasoonS]

I'm experiencing an issue when querying transaction receipts for block 2285347 on Moonbase Alpha. Specifically, the blockNumber field in some transaction receipts does not match the expected block number. This inconsistency is causing problems for us as we rely on accurate block data for our application.

---

Issue Details:

• Block Number Queried: 2285347
• Expected blockNumber in Receipts: 0x22DF23 (Hexadecimal for 2285347)
• Actual blockNumber in Receipts: Mixed between 0x22DF23 and 0x22DF24 (2285347 and 2285348)
• Affected Transactions:
  • 0x006a6843eb35ad35a9ea9a99affa8d81f1ed500253c98cc9c080d84171a0afb3
  • 0x64c102f664eb435206ad4fcb49b526722176bcf74801c79473c3b5b2c281a243
  • 0xf546335453b6e35ce7e236ee873c96ba3a22602b3acc4f45f5d68b33a76d79ca
  • 0x4ed713ccd474fc33d2022a802f064cc012e3e37cd22891d4a89c7ba3d776f2db
  • 0xa5355f86844bb23fe666b10b509543fa377a9e324513eb221e0a2c926a64cae4
  • 0xc14791a3a392018fc3438f39cac1d572e8baadd4ed350e0355d1ca874a169e6a

I have tested this on a few of the available public urpc endpoints.

---

Steps to Reproduce:

1. Use the Following Script:

   #!/bin/bash
   
   # RPC endpoint for Moonbase Alpha
   RPC_ENDPOINT="https://rpc.api.moonbase.moonbeam.network"
   
   # Block number
   BLOCK_NUMBER=2285347
   BLOCK_NUMBER_HEX=$(printf "0x%X" $BLOCK_NUMBER)
   
   # Get block data with transactions included
   BLOCK_DATA=$(curl -s -X POST \
    -H "Content-Type: application/json" \
    --data '{
      "jsonrpc": "2.0",
      "method": "eth_getBlockByNumber",
      "params": ["'"$BLOCK_NUMBER_HEX"'", true],
      "id": 1
    }' \
    "$RPC_ENDPOINT")
   
   # Extract transaction hashes using jq
   TX_HASHES=$(echo "$BLOCK_DATA" | jq -r '.result.transactions[].hash')
   
   # Check if there are any transactions in the block
   if [ -z "$TX_HASHES" ]; then
      echo "No transactions found in block $BLOCK_NUMBER."
      exit 0
   fi
   
   echo "Transaction hashes in block $BLOCK_NUMBER: $TX_HASHES"
   
   # Loop through each transaction hash and get its receipt
   for TX_HASH in $TX_HASHES; do
      echo "Fetching receipt for transaction: $TX_HASH"
      TX_RECEIPT=$(curl -s -X POST \
        -H "Content-Type: application/json" \
        --data '{
          "jsonrpc": "2.0",
          "method": "eth_getTransactionReceipt",
          "params": ["'"$TX_HASH"'"],
          "id": 1
        }' \
        "$RPC_ENDPOINT")
   
      # Check if transaction receipt was retrieved successfully
      if [ -z "$TX_RECEIPT" ] || [ "$(echo "$TX_RECEIPT" | jq -r '.result')" == "null" ]; then
          echo "Error: Failed to retrieve transaction receipt for $TX_HASH."
          continue
      fi
   
      # Print the block number from the transaction receipt
      BLOCK_NUMBER_IN_RECEIPT=$(echo "$TX_RECEIPT" | jq -r '.result.blockNumber')
      echo "Transaction Receipt Block Number: $BLOCK_NUMBER_IN_RECEIPT"
   
      echo "========================================"
   done
   
   # Cross-reference: eth_getBlockReceipts
   echo "Fetching block receipts using eth_getBlockReceipts for block $BLOCK_NUMBER"
   BLOCK_RECEIPTS=$(curl -s -X POST \
    -H "Content-Type: application/json" \
    --data '{
      "jsonrpc": "2.0",
      "method": "eth_getBlockReceipts",
      "params": ["'"$BLOCK_NUMBER_HEX"'"],
      "id": 1
    }' \
    "$RPC_ENDPOINT")
   
   echo "$BLOCK_RECEIPTS" | jq .

(got chatgpt to whip this script up as an example, but should be easy to understand bash)

---

Observed Output:

Transaction hashes in block 2285347: 0x006a6843eb35ad35a9ea9a99affa8d81f1ed500253c98cc9c080d84171a0afb3
0x64c102f664eb435206ad4fcb49b526722176bcf74801c79473c3b5b2c281a243
0xf546335453b6e35ce7e236ee873c96ba3a22602b3acc4f45f5d68b33a76d79ca
0x4ed713ccd474fc33d2022a802f064cc012e3e37cd22891d4a89c7ba3d776f2db
0xa5355f86844bb23fe666b10b509543fa377a9e324513eb221e0a2c926a64cae4
0xc14791a3a392018fc3438f39cac1d572e8baadd4ed350e0355d1ca874a169e6a
Fetching receipt for transaction: 0x006a6843eb35ad35a9ea9a99affa8d81f1ed500253c98cc9c080d84171a0afb3
Transaction Receipt Block Number: 0x22df24
========================================
Fetching receipt for transaction: 0x64c102f664eb435206ad4fcb49b526722176bcf74801c79473c3b5b2c281a243
Transaction Receipt Block Number: 0x22df24
========================================
Fetching receipt for transaction: 0xf546335453b6e35ce7e236ee873c96ba3a22602b3acc4f45f5d68b33a76d79ca
Transaction Receipt Block Number: 0x22df24
========================================
Fetching receipt for transaction: 0x4ed713ccd474fc33d2022a802f064cc012e3e37cd22891d4a89c7ba3d776f2db
Transaction Receipt Block Number: 0x22df23
========================================
Fetching receipt for transaction: 0xa5355f86844bb23fe666b10b509543fa377a9e324513eb221e0a2c926a64cae4
Transaction Receipt Block Number: 0x22df24
========================================
Fetching receipt for transaction: 0xc14791a3a392018fc3438f39cac1d572e8baadd4ed350e0355d1ca874a169e6a
Transaction Receipt Block Number: 0x22df23
========================================
Fetching block receipts using eth_getBlockReceipts for block 2285347
{
  "jsonrpc": "2.0",
  "id": 1,
  "result": [ ... ]
}

---

Problem Description:

• Inconsistency in blockNumber: When fetching transaction receipts for transactions in block 2285347, some transactions report their blockNumber as 0x22df23 (2285347), while others report 0x22df24 (2285348).
• Expected Behavior: All transaction receipts for transactions in block 2285347 should have a blockNumber of 0x22df23.
• Impact: This discrepancy causes issues in our application, as we rely on the accuracy of the blockNumber field to process and validate transactions in our indexer.

---

Analysis:

• Transactions with blockNumber as 0x22df23:
  • 0x4ed713ccd474fc33d2022a802f064cc012e3e37cd22891d4a89c7ba3d776f2db
  • 0xc14791a3a392018fc3438f39cac1d572e8baadd4ed350e0355d1ca874a169e6a
• Transactions with blockNumber as 0x22df24:
  • 0x006a6843eb35ad35a9ea9a99affa8d81f1ed500253c98cc9c080d84171a0afb3
  • 0x64c102f664eb435206ad4fcb49b526722176bcf74801c79473c3b5b2c281a243
  • 0xf546335453b6e35ce7e236ee873c96ba3a22602b3acc4f45f5d68b33a76d79ca
  • 0xa5355f86844bb23fe666b10b509543fa377a9e324513eb221e0a2c926a64cae4

[RomarQ]

Hi @JasoonS,

I did have a look at this issue and have been able to identify the problem, it was a bug introduced by this PR in the frontier repository: https://github.com/polkadot-evm/frontier/pull/638

Thankfully it was detected in time and was fixed before going live on moonriver and moonbeam.

Important to note that those eth transactions were never replayed, it was a indexing bug on pallet_ethereum logic, where after the runtime upgrade from version 1504 to 1603, 6 transactions from the previous block were also included as being part of the next block.

A possible solution is to add a moonbase specific patch that blacklists those transactions when querying block 2285348, will work on it.

[RomarQ]

Hi @JasoonS,

The issue you mentioned above is described here: https://docs.moonbeam.network/builders/build/historical-updates/#ethereum-transactions-duplicated-in-storage

We will probably not include a patch specifically for this since it would not solve the inconsistency completely without an hard-fork (The patch we currently have would only filter the duplicated transactions from the block, but would not fix the receipts_root and transactions_root fields in the block header).

On your app, you will have to add some custom logic to handle this insistency.

[JasoonS]

Hi @RomarQ - Thank you for the detailed answer, that now makes sense. We have managed to disable validation in the meantime, and thankfully it is a testnet, so not super mission critical.

Once again, massive thank you for your time and attention.