scroll-tech / go-ethereum

Scroll's fork of the official Go implementation of the Ethereum protocol
GNU Lesser General Public License v3.0
470 stars 271 forks source link

feat: add toolkit for exporting and transforming missing block header fields #903

Open jonastheis opened 2 months ago

jonastheis commented 2 months ago

1. Purpose or design rationale of this PR

We are using the Clique consensus in Scroll L2. Amongst others, it requires the following header fields:

However, these fields are currently not stored on L1/DA and we're planning to add them in a future upgrade. In order for nodes to be able to reconstruct the correct block hashes when only reading data from L1, we need to provide the historical values of these fields to these nodes through a separate file.

This toolkit provides commands to export the missing fields, deduplicate the data and create a file with the missing fields that can be used to reconstruct the correct block hashes when only reading data from L1.

Analysis of data

Mainnet until block 7455960

--------------------------------------------------
Difficulty 1: 1
Difficulty 2: 7455960
Vanity: d883050320846765746888676f312e32302e31856c696e757800000000000000
Vanity: d883050406846765746888676f312e32312e31856c696e757800000000000000
Vanity: d883050408846765746888676f312e32312e31856c696e757800000000000000
Vanity: d883050103846765746888676f312e31392e31856c696e757800000000000000
Vanity: d883050108846765746888676f312e31392e31856c696e757800000000000000
Vanity: d883050203846765746888676f312e32302e31856c696e757800000000000000
Vanity: d883050400846765746888676f312e32312e31856c696e757800000000000000
Vanity: d883050508846765746888676f312e32312e31856c696e757800000000000000
Vanity: d883050000846765746888676f312e31392e31856c696e757800000000000000
Vanity: d88305030b846765746888676f312e32302e31856c696e757800000000000000
Vanity: d883050402846765746888676f312e32312e31856c696e757800000000000000
Vanity: d88305011e846765746888676f312e32302e31856c696e757800000000000000
Vanity: d883050206846765746888676f312e32302e31856c696e757800000000000000
Vanity: d883050506846765746888676f312e32312e31856c696e757800000000000000
Vanity: d88305050a846765746888676f312e32312e31856c696e757800000000000000
Vanity: d883050107846765746888676f312e31392e31856c696e757800000000000000
Vanity: d883050311846765746888676f312e32302e31856c696e757800000000000000
Vanity: d883050500846765746888676f312e32312e31856c696e757800000000000000
Vanity: d88305030c846765746888676f312e32302e31856c696e757800000000000000
Vanity: d883050106846765746888676f312e31392e31856c696e757800000000000000
Vanity: d88305010a846765746888676f312e31392e31856c696e757800000000000000
Vanity: d883050300846765746888676f312e32302e31856c696e757800000000000000
Vanity: 4c61206573746f6e7465636f206573746173206d616c6665726d6974612e0000
Vanity: d883050001846765746888676f312e31392e31856c696e757800000000000000
Vanity: d88305010b846765746888676f312e31392e31856c696e757800000000000000
Vanity: d883050003846765746888676f312e31392e31856c696e757800000000000000
Vanity: d883050109846765746888676f312e31392e31856c696e757800000000000000
Vanity: d883050121846765746888676f312e32302e31856c696e757800000000000000
SealLen 85 bytes: 249
SealLen 65 bytes: 7455712
--------------------------------------------------
Unique values seen in the headers file (last seen block: 7455960):
Distinct count: Difficulty:2, Vanity:28, SealLen:2
--------------------------------------------------

Sepolia until block 5422047

--------------------------------------------------
Difficulty 2: 5422047
Difficulty 1: 1
Vanity: d88305031a846765746888676f312e32302e31856c696e757800000000000000
Vanity: d883040404846765746888676f312e31392e31856c696e757800000000000000
Vanity: d883050312846765746888676f312e32302e31856c696e757800000000000000
Vanity: d883040338846765746888676f312e31392e31856c696e757800000000000000
Vanity: d883040339846765746888676f312e31392e31856c696e757800000000000000
Vanity: d88304040e846765746888676f312e31392e31856c696e757800000000000000
Vanity: d883040500846765746888676f312e31392e31856c696e757800000000000000
Vanity: d883050105846765746888676f312e31392e31856c696e757800000000000000
Vanity: 0000000000000000000000000000000000000000000000000000000000000000
Vanity: d883040320846765746888676f312e31392e31856c696e757800000000000000
Vanity: d883050400846765746888676f312e32312e31856c696e757800000000000000
Vanity: d883050408846765746888676f312e32312e31856c696e757800000000000000
Vanity: d883040325846765746888676f312e31392e31856c696e757800000000000000
Vanity: d883050121846765746888676f312e32302e31856c696e757800000000000000
Vanity: d88305030a846765746888676f312e32302e31856c696e757800000000000000
Vanity: d88305030c846765746888676f312e32302e31856c696e757800000000000000
Vanity: d88304033b846765746888676f312e31392e31856c696e757800000000000000
Vanity: d883040407846765746888676f312e31392e31856c696e757800000000000000
Vanity: d883050102846765746888676f312e31392e31856c696e757800000000000000
Vanity: d883050300846765746888676f312e32302e31856c696e757800000000000000
Vanity: d883050508846765746888676f312e32312e31856c696e757800000000000000
Vanity: d88304032a846765746888676f312e31392e31856c696e757800000000000000
Vanity: d88304033a846765746888676f312e31392e31856c696e757800000000000000
Vanity: d883050316846765746888676f312e32302e31856c696e757800000000000000
Vanity: d88304040b846765746888676f312e31392e31856c696e757800000000000000
Vanity: d88304040c846765746888676f312e31392e31856c696e757800000000000000
Vanity: d883050206846765746888676f312e32302e31856c696e757800000000000000
Vanity: d883050320846765746888676f312e32302e31856c696e757800000000000000
Vanity: d88305011e846765746888676f312e32302e31856c696e757800000000000000
Vanity: d883050317846765746888676f312e32302e31856c696e757800000000000000
Vanity: d883050318846765746888676f312e32302e31856c696e757800000000000000
Vanity: d88305031e846765746888676f312e32302e31856c696e757800000000000000
Vanity: d883050403846765746888676f312e32312e31856c696e757800000000000000
Vanity: d883050404846765746888676f312e32312e31856c696e757800000000000000
Vanity: d883050407846765746888676f312e32312e31856c696e757800000000000000
Vanity: d88304033e846765746888676f312e31392e31856c696e757800000000000000
Vanity: d88304033f846765746888676f312e31392e31856c696e757800000000000000
Vanity: d883050506846765746888676f312e32312e31856c696e757800000000000000
Vanity: d88304032b846765746888676f312e31392e31856c696e757800000000000000
Vanity: d88304040f846765746888676f312e31392e31856c696e757800000000000000
Vanity: d883050311846765746888676f312e32302e31856c696e757800000000000000
Vanity: d883050003846765746888676f312e31392e31856c696e757800000000000000
Vanity: d883050200846765746888676f312e32302e31856c696e757800000000000000
Vanity: d883050114846765746888676f312e32302e31856c696e757800000000000000
Vanity: d883040328846765746888676f312e31392e31856c696e757800000000000000
Vanity: d883040402846765746888676f312e31392e31856c696e757800000000000000
Vanity: d88305010d846765746888676f312e32302e31856c696e757800000000000000
Vanity: d88304031d846765746888676f312e31392e31856c696e757800000000000000
Vanity: d883040321846765746888676f312e31392e31856c696e757800000000000000
Vanity: d883050001846765746888676f312e31392e31856c696e757800000000000000
Vanity: d883050107846765746888676f312e31392e31856c696e757800000000000000
Vanity: d88304031f846765746888676f312e31392e31856c696e757800000000000000
Vanity: d883040336846765746888676f312e31392e31856c696e757800000000000000
SealLen 85 bytes: 181
SealLen 65 bytes: 5421867
--------------------------------------------------
Unique values seen in the headers file (last seen block: 5422047):
Distinct count: Difficulty:2, Vanity:53, SealLen:2
--------------------------------------------------

2. PR title

Your PR title must follow conventional commits (as we are doing squash merge for each PR), so it must start with one of the following types:

3. Deployment tag versioning

Has the version in params/version.go been updated?

4. Breaking change label

Does this PR have the breaking-change label?

HAOYUatHZ commented 2 months ago

why sometimes use "-" sometimes use "_" in the path?

jonastheis commented 2 months ago

missing_header_fields is a package within l2geth, which will host some other functionality to read the missing header file later on and will be used within l2geth. export-headers-toolkit is a standalone, separate module that doesn't need to run in the context of l2geth.