Gnosis Archive + Lighthouse building 0 gas blocks

[CosmiCloud]

System information

Erigon version: ./erigon --version thorax/erigon:v2.59.3

OS & Version: Windows/Linux/OSX Linux

Commit hash:

Erigon Command (with flags/config): command:

• erigon
• --chain=gnosis
• --datadir=/opt/data/erigon
• --authrpc.addr=0.0.0.0
• --authrpc.vhosts=*
• --authrpc.port=8551
• --authrpc.jwtsecret=/engine.jwt
• --rpc.returndata.limit=1000000
• --ws
• --http
• --http.vhosts=*
• --http.corsdomain=*
• --http.addr=0.0.0.0
• --http.port=8545
• --http.api=engine,net,eth,web3
• --metrics
• --metrics.addr=0.0.0.0
• --metrics.port=6060
• --db.pagesize=16K
• --db.size.limit=8TB

Consensus Layer: Lighthouse

Consensus Layer Command (with flags/config): command:

• lighthouse
• bn
• --debug-level=info
• --network=gnosis
• --execution-endpoint=http://stereum-1f58a8fc-fc07-9e7f-08e8-c9b59b21c226:8551
• --execution-jwt=/engine.jwt
• --eth1-blocks-per-log-query=150
• --datadir=/opt/app/beacon
• --http
• --http-address=0.0.0.0
• --metrics
• --metrics-address=0.0.0.0
• --disable-upnp
• --validator-monitor-auto
• --slasher
• --slasher-dir=/opt/app/slasher
• --port=9000
• --enr-tcp-port=9000
• --enr-udp-port=9000
• --disable-enr-auto-update=false
• --reconstruct-historic-states
• --genesis-backfill
• --checkpoint-sync-url=https://checkpoint.gnosischain.com/

command:

• lighthouse
• vc
• --enable-doppelganger-protection
• --debug-level=debug
• --network=gnosis
• --beacon-nodes=http://stereum-d6244241-8594-ae74-cb05-5f5e46454bca:5052
• --suggested-fee-recipient=0xbe6001bEf7251f5975cD85570b0C5F04Ba0DfDd9
• --datadir=/opt/app/validator
• --init-slashing-protection
• --graffiti-file=/opt/app/graffitis/graffitis.yaml
• --metrics
• --metrics-address=0.0.0.0
• --http
• --http-address=0.0.0.0
• --http-port=5062
• --unencrypted-http-transport
• --builder-proposals

Chain/Network: Gnosis

Expected behaviour

When a validator proposes a block, the block is NOT built with 0% gas used.

Actual behaviour

When a validator proposes a block, the block is built with 0% gas used. https://gnosischa.in/address/0xbe6001bef7251f5975cd85570b0c5f04ba0dfdd9#blocks

Example: slot 15060940, block 33670912 Below are the logs of the node from the start of the sync to present day.: 0 gas blocks logs.zip

EC: {"log":"[INFO] [04-28|09:53:58.344] [ForkChoiceUpdated] BlockBuilder added payload=3\n","stream":"stderr","time":"2024-04-28T09:53:58.344849359Z"} {"log":"[INFO] [04-28|09:53:58.344] Building block... \n","stream":"stderr","time":"2024-04-28T09:53:58.344974321Z"} {"log":"[INFO] [04-28|09:53:58.355] [1/6 MiningCreateBlock] Start mine block=33670912 baseFee=8 gasLimit=17016600\n","stream":"stderr","time":"2024-04-28T09:53:58.358164657Z"} {"log":"[INFO] [04-28|09:53:58.392] Built block hash=0xb0b99abef320240b1f57b83be76ac0361b5c5f3169ae61dd53fc8d0b45e467e0 height=33670912 txs=0 gas used %=0.000 time=47.256791ms\n","stream":"stderr","time":"2024-04-28T09:53:58.392291424Z"} {"log":"[INFO] [04-28|09:53:59.803] [ForkChoiceUpdated] duplicate build request \n","stream":"stderr","time":"2024-04-28T09:53:59.803956883Z"} {"log":"[INFO] [04-28|09:54:00.015] Received GetPayloadV3 payloadId=3\n","stream":"stderr","time":"2024-04-28T09:54:00.015934071Z"} {"log":"[INFO] [04-28|09:54:00.141] [NewPayload] Handling new payload height=33670912 hash=0xb0b99abef320240b1f57b83be76ac0361b5c5f3169ae61dd53fc8d0b45e467e0\n","stream":"stderr","time":"2024-04-28T09:54:00.141485727Z"} {"log":"[INFO] [04-28|09:54:00.186] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload) \n","stream":"stderr","time":"2024-04-28T09:54:00.186587784Z"} {"log":"[INFO] [04-28|09:54:00.190] RPC Daemon notified of new headers from=33670911 to=33670912 amount=1 hash=0xb0b99abef320240b1f57b83be76ac0361b5c5f3169ae61dd53fc8d0b45e467e0 header sending=9.839µs log sending=172ns\n","stream":"stderr","time":"2024-04-28T09:54:00.191072404Z"} {"log":"[INFO] [04-28|09:54:00.190] head updated hash=0xb0b99abef320240b1f57b83be76ac0361b5c5f3169ae61dd53fc8d0b45e467e0 number=33670912\n","stream":"stderr","time":"2024-04-28T09:54:00.191084737Z"} {"log":"[INFO] [04-28|09:54:05.373] [NewPayload] Handling new payload height=33670913 hash=0x14ef6972c89612311be66b4d068c91385708f839c0b067fb422652944b64beda\n","stream":"stderr","time":"2024-04-28T09:54:05.373936273Z"} {"log":"[INFO] [04-28|09:54:05.492] [updateForkchoice] Fork choice update: flushing in-memory state (built by previous newPayload) \n","stream":"stderr","time":"2024-04-28T09:54:05.492990609Z"} {"log":"[INFO] [04-28|09:54:05.806] RPC Daemon notified of new headers from=33670912 to=33670913 amount=1 hash=0x14ef6972c89612311be66b4d068c91385708f839c0b067fb422652944b64beda header sending=11.514µs log sending=159ns\n","stream":"stderr","time":"2024-04-28T09:54:05.806837333Z"}

BN: {"log":"Apr 28 09:53:55.009 INFO Prepared beacon proposer parent_root: 0x2e76a424a99f740601544392c8f89fbef4f5231dda0c555e2d44267ffd9a031f, validator: 241502, prepare_slot: 15060940, service: beacon\n","stream":"stderr","time":"2024-04-28T09:53:55.009716191Z"} {"log":"Apr 28 09:53:55.749 INFO New block received root: 0xa291c49f6278d3b64b714c9898faf913a68e6235121633ba347f0d0897dbdde0, slot: 15060939\n","stream":"stderr","time":"2024-04-28T09:53:55.749288475Z"} {"log":"Apr 28 09:53:55.935 INFO Prepared beacon proposer parent_root: 0xa291c49f6278d3b64b714c9898faf913a68e6235121633ba347f0d0897dbdde0, validator: 241502, prepare_slot: 15060940, service: beacon\n","stream":"stderr","time":"2024-04-28T09:53:55.935774805Z"} {"log":"Apr 28 09:53:57.501 INFO Synced slot: 15060939, block: 0xa291…dde0, epoch: 941308, finalized_epoch: 941306, finalized_root: 0x2dfe…9804, exec_hash: 0xbc0f…f144 (verified), peers: 77, service: slot_notifier\n","stream":"stderr","time":"2024-04-28T09:53:57.501213497Z"} {"log":"Apr 28 09:54:00.098 INFO Reconstructing a full block using a local payload, block_hash: 0xb0b99abef320240b1f57b83be76ac0361b5c5f3169ae61dd53fc8d0b45e467e0\n","stream":"stderr","time":"2024-04-28T09:54:00.098301415Z"} {"log":"Apr 28 09:54:00.099 INFO Signed block published to network via HTTP API, publish_delay: 1.631163ms, slot: 15060940\n","stream":"stderr","time":"2024-04-28T09:54:00.099936237Z"} {"log":"Apr 28 09:54:00.158 INFO Valid block from HTTP API slot: 15060940, proposer_index: 241502, root: 0x0287…8135, block_delay: 98.214205ms\n","stream":"stderr","time":"2024-04-28T09:54:00.158954491Z"} {"log":"Apr 28 09:54:00.158 INFO Block from API validator: 241502, src: api, slot: 15060940, delay: 98, root: 0x02876186da2cb738ad90ff1cff4f7b5e0fdb645d930acd681504f05c5e418135, service: val_mon, service: beacon\n","stream":"stderr","time":"2024-04-28T09:54:00.158970827Z"}

VC: {"log":"Apr 28 09:54:00.004 DEBG Published proposer preparation count: 15, service: preparation\n","stream":"stderr","time":"2024-04-28T09:54:00.004413301Z"} {"log":"Apr 28 09:54:00.006 DEBG Downloaded proposer duties num_relevant_duties: 1, dependent_root: 0x3cb9…799a, service: duties\n","stream":"stderr","time":"2024-04-28T09:54:00.006565604Z"} {"log":"Apr 28 09:54:00.006 DEBG Broadcast attestation subscriptions count: 6, service: duties\n","stream":"stderr","time":"2024-04-28T09:54:00.006931568Z"} {"log":"Apr 28 09:54:00.007 INFO Requesting unsigned block slot: 15060940, service: block\n","stream":"stderr","time":"2024-04-28T09:54:00.007487426Z"} {"log":"Apr 28 09:54:00.082 INFO Received unsigned block slot: 15060940, service: block\n","stream":"stderr","time":"2024-04-28T09:54:00.082339222Z"} {"log":"Apr 28 09:54:00.085 INFO Publishing signed block signing_time_ms: 3, slot: 15060940, service: block\n","stream":"stderr","time":"2024-04-28T09:54:00.085579862Z"} {"log":"Apr 28 09:54:00.235 INFO Successfully published block slot: 15060940, graffiti: Some(\"Stake $TRAC with OThub.io!\"), attestations: 74, deposits: 0, block_type: Blinded, service: block\n","stream":"stderr","time":"2024-04-28T09:54:00.235912533Z"} {"log":"Apr 28 09:54:01.667 DEBG No local validators in current sync committee, slot: 15060940, service: sync_committee\n","stream":"stderr","time":"2024-04-28T09:54:01.667753973Z"} {"log":"Apr 28 09:54:01.667 DEBG Fetching subscription duties current_slot: 15060940, duty_slot: 15060940, service: sync_committee\n","stream":"stderr","time":"2024-04-28T09:54:01.667800091Z"} {"log":"Apr 28 09:54:01.667 DEBG No sync subscriptions to send slot: 15060940, service: sync_committee\n","stream":"stderr","time":"2024-04-28T09:54:01.667809015Z"} {"log":"Apr 28 09:54:01.675 INFO Successfully published attestations type: unaggregated, slot: 15060940, committee_index: 26, head_block: 0x02876186da2cb738ad90ff1cff4f7b5e0fdb645d930acd681504f05c5e418135, validator_indices: [241501], count: 1, service: attestation\n","stream":"stderr","time":"2024-04-28T09:54:01.675994902Z"} {"log":"Apr 28 09:54:02.392 DEBG Measured BN latency latency: 0, node: http://stereum-d6244241-8594-ae74-cb05-5f5e46454bca:5052/\n","stream":"stderr","time":"2024-04-28T09:54:02.392426618Z"} {"log":"Apr 28 09:54:02.500 INFO Connected to beacon node(s) synced: 1, available: 1, total: 1, service: notifier\n","stream":"stderr","time":"2024-04-28T09:54:02.500647609Z"} {"log":"Apr 28 09:54:02.500 INFO All validators active slot: 15060940, epoch: 941308, total_validators: 15, active_validators: 15, current_epoch_proposers: 1, service: notifier\n","stream":"stderr","time":"2024-04-28T09:54:02.500729552Z"}

Steps to reproduce the behaviour

Backtrace

[backtrace]

[CosmiCloud]

Restarted Erigon, then lighthouse. Next block proposed DID have fees.