Op-erigon Base node lags on NewHeads

[Igorarg91]

Description: We are facing increased delay up to 17 sec between receiving newHeads and the actual timestamp of each blocks on our Base protocol op-erigon client. What could be the reason and how to decrease this lag?

System information

Network protocol: base blockchain-network: mainnet blockchain-client: op-erigon blockchain-debug-trace: 'true' blockchain-mode: archive op-erigon-image: testinprod/op-erigon:2.60.1-0.6.3-amd64 op-node-image: ghcr.io/base-org/node:v0.9.0

Kubernetes node information STATUS VERSION OS-IMAGE KERNEL-VERSION CONTAINER Ready v1.27.9-eks-5e0fdde Amazon Linux 2 5.10.210-201.852.amzn2.x86_64 containerd://1.7.11

kubernetes version Client Version: v1.27.3 Kustomize Version: v5.0.1 Server Version: v1.27.13-eks-3af4770

op-erigon container config

• name: op-erigon image: testinprod/op-erigon:2.60.1-0.6.3-amd64 command:
  • erigon
  • '--datadir=/home/node/data'
  • '--db.pagesize=4096'
  • '--ethash.dagdir=/home/node/data/dag'
  • '--db.size.limit=8TB'
  • '--chain=base-mainnet'
  • '--networkid=8453'
  • '--authrpc.jwtsecret=/home/node/jwt-secret.txt'
  • '--authrpc.vhosts=*'
  • '--http'
  • '--http.addr=0.0.0.0'
  • '--http.corsdomain=null'
  • '--http.vhosts=*'
  • '--http.api=eth,erigon,net,web3,debug,trace'
  • '--ws'
  • '--rpc.batch.limit=1000'
  • '--rpc.returndata.limit=150000'
  • '--rollup.sequencerhttp=https://mainnet-sequencer.base.org'
  • '--rollup.historicalrpc='
  • '--rollup.historicalrpctimeout=5s'
  • '--rpc.txfeecap=100'
  • '--maxpeers=0'
  • '--nodiscover'
  • '--rollup.disabletxpoolgossip=true'
  • '--metrics'
  • '--metrics.addr=0.0.0.0'
  • '--healthcheck'
  • '--private.api.addr=0.0.0.0:9090'
  • '--private.api.ratelimit=63744'
  • '--db.size.limit=8TB'
  • '--rpc.batch.concurrency=4'
  • '--db.read.concurrency=4096'
  • '--rpc.batch.limit=200'
  • '--rpc.returndata.limit=200000'

op-node container config containers:

• name: op-node image: ghcr.io/base-org/node:v0.9.0 command:
  • /app/op-node
  • '--network=base-mainnet'

  • - --l1=http://erigon-ethereum-mainnet-xxx.default.svc.cluster.local:8545

  • - --l1.beacon=http://erigon-ethereum-mainnet-xxx.default.svc.cluster.local:5052

  • '--l2=ws://localhost:8551'
  • '--l2.jwt-secret=/home/node/jwt-secret.txt'
  • '--rpc.addr=0.0.0.0'
  • '--rpc.port=9545'
  • '--p2p.priv.path=/home/node/p2p-private-key.txt'
  • '--p2p.peerstore.path=memory'
  • '--p2p.discovery.path=memory'
  • '--metrics.enabled'
  • '--l1.trustrpc=true'

Erigon version: ./erigon --version

OS & Version: Windows/Linux/OSX

Commit hash:

Erigon Command (with flags/config):

Consensus Layer:

Consensus Layer Command (with flags/config):

Chain/Network:

Expected behaviour

Lag on NewHeads between original blocks created timestamp and timestamp on our nodes is below 1 second.

Actual behaviour

Sometimes delay reaches up to 17 sec between receiving newHeads and the actual timestamp of each blocks on our Base node.

5 Worst Latencies for Base Node: Block Number My Timestamp Original Timestamp Delta 2353 16615734 2024-07-03 12:33:52 2024-07-03 15:33:35 17.0 2355 16615735 2024-07-03 12:33:52 2024-07-03 15:33:37 15.0 2356 16615736 2024-07-03 12:33:52 2024-07-03 15:33:39 13.0 2357 16615737 2024-07-03 12:33:52 2024-07-03 15:33:41 11.0 2358 16615738 2024-07-03 12:33:52 2024-07-03 15:33:43 9.0

Steps to reproduce the behaviour

Using this script to calculate lag of newHeads.