[Proposal] Add `ExecutionPayloadHeader` to `BuilderSubmitBlockRequest` for builder submissions

[michaelneuder]

Proposal

The current API defines the request data type for the builder blocks submissions as a BuilderSubmitBlockRequest.

type BuilderSubmitBlockRequest struct {
    Signature        Signature         `json:"signature" ssz-size:"96"`
    Message          *BidTrace         `json:"message"`
    ExecutionPayload *ExecutionPayload `json:"execution_payload"`
}

This works well for situations where the relay expects the entire execution payload from the builder and parses this data before continuing on with the block processing only once the entire payload is downloaded. However, the ExecutionPayload can be quite large. The figure below shows the size in bytes of 10,000 ExecutionPayloads received by the ultra sound relay.

With optimistic relaying (see "An optimistic weekend") transmitting the bytes of the ExecutionPayload from the builder to the relay, especially when the builder is not geographically close to the relay, is enough to bottleneck the entire block submission for builders. The figure below shows the decode duration, for 1 million optimistic and non-optimistic blocks. The bimodality of the data shows relays the importance of continental collocation with the relay.

Another way of looking at it is the proportion of the submission duration that is spent receiving and decoding the payload.

On average 62% of optimistic relay submission duration is spent receiving the ExecutionPayload over the network. As part of the optimistic roadmap the ultra sound relay aims to parse only the header from the builder message on the fast path, which expedites the speed at which a builder bid can become eligible to win the auction. To send a bid to the proposer, the relay must have the full ExecutionPayloadHeader that the proposer signs over to commit to a specific bid. We propose expanding the BuilderSubmitBlockRequest with an optional ExecutionPayloadHeader:

type BuilderSubmitBlockRequest struct {
    Signature               Signature                `json:"signature" ssz-size:"96"`
    Message                 *BidTrace                `json:"message"`
    ExecutionPayloadHeader  *ExecutionPayloadHeader  `json:"execution_payload_header"`
    ExecutionPayload        *ExecutionPayload        `json:"execution_payload"`
}

The relay functionality can be configured based on the availability of this header. For the ultra sound relay, an optimistic builder can send a message with the header before the payload, which will allow us to mark the bid as eligible and receive the remaining packets of the payload on the slow path. The also is compatible with future versions of relays, which might not ever receive the full ExecutionPayload, but instead rely on the builder to publish the signed block.

Byte packing

Ideally, we want the header + bid details to fit fully into a single 1500 byte Ethernet packet. This is the definition of the header type itself, which is 32+20+32+32+256+32+8+8+8+8+32+32+32+32=564 bytes:

type ExecutionPayloadHeader struct {
    ParentHash       Hash      `json:"parent_hash" ssz-size:"32"`  
    FeeRecipient     Address   `json:"fee_recipient" ssz-size:"20"`
    StateRoot        Root      `json:"state_root" ssz-size:"32"`
    ReceiptsRoot     Root      `json:"receipts_root" ssz-size:"32"`
    LogsBloom        Bloom     `json:"logs_bloom" ssz-size:"256"`
    Random           Hash      `json:"prev_randao" ssz-size:"32"`
    BlockNumber      uint64    `json:"block_number,string"`
    GasLimit         uint64    `json:"gas_limit,string"`
    GasUsed          uint64    `json:"gas_used,string"`
    Timestamp        uint64    `json:"timestamp,string"`
    ExtraData        ExtraData `json:"extra_data" ssz-max:"32"`
    BaseFeePerGas    U256Str   `json:"base_fee_per_gas" ssz-size:"32"`
    BlockHash        Hash      `json:"block_hash" ssz-size:"32"`
    TransactionsRoot Root      `json:"transactions_root" ssz-size:"32"`
}

The bid trace itself is 8+32+32+48+48+20+8+8+32=236 bytes:

type BidTrace struct {
    Slot                 uint64    `json:"slot,string"`
    ParentHash           Hash      `json:"parent_hash" ssz-size:"32"`
    BlockHash            Hash      `json:"block_hash" ssz-size:"32"`
    BuilderPubkey        PublicKey `json:"builder_pubkey" ssz-size:"48"`
    ProposerPubkey       PublicKey `json:"proposer_pubkey" ssz-size:"48"`
    ProposerFeeRecipient Address   `json:"proposer_fee_recipient" ssz-size:"20"`
    GasLimit             uint64    `json:"gas_limit,string"`
    GasUsed              uint64    `json:"gas_used,string"`
    Value                U256Str   `json:"value" ssz-size:"32"`
}

Along with the signature (96 bytes), the first 3 fields of the new BuilderSubmitBlockRequest are only 896 bytes, which is will within the MTU of a single packet. This will help builders and relays limit the impact of network latency on their submission performance.

NOTE: the flashbots capella changes introduce 2 new fields into the bid trace, each is only 8 bytes, see BidTraceV2. Additionally, there is a new withdrawls_root in the spec definition of the ExecutionPayloadHeader, which adds another 32 bytes. That brings the capella bid + signature + header size to 912 bytes.

[metachris]

I think it makes sense to have the header with tx root as part of the submission, rather than forcing the relay to compute the header after submission.

Wonder whether, instead of ExecutionPayload, it wouldn't be cleaner to have just a txs field, the other information is repetitive.

Sidenote: your bytes calculation is only correct if we assume the submission is SSZ encoded. If it's JSON then it's roughly double that, because of the field names.