Block produced with incorrect Fee Recipient 0x0000000000000000000000000000000000000001

[PacoBits]

Our setup has produced a block with an incorrect fee recipient: https://etherscan.io/block/15616137 We have been producing blocks correctly and after that incident it is still producing blocks with the fee recipient correctly. We have a 3-layer configuration with the validator client, the Beacon node and the execution client running separately.

This issue is not related to teku restart. Its RARE and not replicable at this moment.

These are our versions and fee recipent setup:

Validator set up - version: Lighthouse/v3.1.0-aa022f4 --graffiti "Block by Stakely Lido09" --suggested-fee-recipient 0x388C818CA8B9251b393131C08a736A67ccB19297

Beacon set up - _teku/v22.9.1/linux-x8664/-eclipseadoptium-openjdk64bitservervm-java-17 "--validators-graffiti='Block by Stakely.io - btk01'", "--validators-proposer-default-fee-recipient=0x7188D41ea11aBf3e95040ba1823E861A62644AC2"

Execution Client - Erigon/v2022.09.3-stable-32bd69e5/linux-amd64/go1.18.3

I attached logs from the time the block produced. The most complete log is the Erigon log, since it is running in debug mode.

Valc09_26092022.log erigon_26092022.log Lighthouse_Beacon_26092022.log teku_26092022.log

[tbenr]

Hello, the logs arrive around Sep 26 02:50:53 (UTC) but the block production you are referring to is at around Sep 26 07:44:00 (UTC)

Can you provide the corresponding logs?

Are you using --log-destination=CONSOLE in your docker\docker-compose file for teku?

[tbenr]

I noticed the feeRecipient address ends with 1. In teku, when something goes wrong and we cannot determine the fee recipient we use the all zeros burn address. So that feeRecipient must come from Lighthouse or Erigon

[PacoBits]

Sorry, I have updated the logs. We have identified the problem.

The block was generated by erigon-teku correctly and sent to the validator, but at that moment there was a disconnection of teku and the validator: Validator client log - disconected beacon: node09 | Sep 26 07:44:29.001 INFO Connected to beacon node(s) synced: 2, available: 2, total: 3, service: notifier

So the validator client continues the publication in the fallback beacon (Lighthouse beacon). This beacon does not have a default fee recipient set due to a problem.

Validator client log - publised block and reconnected to teku:

node09         | Sep 26 07:44:29.001 INFO All validators active                   slot: 4779520, epoch: 149360, total_validators: 500, active_validators: 500, current_epoch_proposers: 1, service: notifier
node09         | Sep 26 07:44:30.208 INFO Successfully published block            slot: 4779520, graffiti: Some("Block by Stakely Lido09"), attestations: 128, deposits: 0, block_type: Full, service: block
node09         | Sep 26 07:44:34.023 INFO Connected to beacon node                endpoint: httpxxx://x.x.x.x:xxxx/, version: teku/v22.9.1/linux-x86_64/-eclipseadoptium-openjdk64bitservervm-java-17

We can see Lighthouse beacon log the problem:

lighthouse_beacon_node_1 | Sep 26 07:44:27.178 CRIT Fee recipient unknown proposer_index: 366592, msg: the suggested_fee_recipient was unknown during block production. a junk address was used, rewards were lost! check the --suggested-fee-recipient flag and VC configuration., service: exec

So we need to solve the problems with fee-recipient on lighthouse and redirect the issue to them.

[PacoBits]

The problem was because the Lighthouse validator client does not send the proposer preparation information to the fallback beacon node, so if an interruption occurs while preparing the block, this problem happens.
Lighthouse team is working on it. Thanks for support. Close the issue