RFP Proposal: Tool for applying test vectors from Ethereum on FEVM

[FroghubMan]

RFP Proposal: Tool for applying test vectors from Ethereum on FEVM

Name of Project: Tool for applying test vectors from Ethereum on FEVM

Link to RFP: fvm-ethereum-test-vectors

RFP Category: devtools-libraries

Proposer: FroghubMan

Do you agree to open source all work you do on behalf of this RFP and dual-license under MIT and APACHE2 licenses?: Yes

Project Description

Technical Approach:

1. For some reasons (signatures, contracts involving some long history, etc.), transactions on Ethereum cannot be fully replayed on the testnet. So we verify FEVM in this way: before replaying a transaction, we will import the transaction into the FEVM of the state (usually there are multiple contracts) involved in Ethereum, and then execute the transaction, In the end we only compare the slots that were modified in the execution of this transaction, not all the slots in the storage of all contracts. for comparison (this involves OpCode SLOAD/SSTORE).
2. Collect the Stack Memory Storage corresponding to the transaction from Ethereum, which contains the data read by SLOAD and the data written by SSTORE, which is used to generate the Test Vector JSON file.
3. FEVM consumes Test Vector. It involves migrating SLOAD data to BlockStore and intercepting SSTORE data, by comparing it with the expected SSTORE data in the Test Vector to get a test report.

Development Roadmap

Milestone1:

1. Fetch and store on the order of hundreds of contracts from Ethereum mainnet.
2. Generate the Test Vectors of thousands of transactions sent to these contracts on Ethereum in the past.

Milestone2: FEVM Consumes Test Vector and generate test result.

Documentation, Education, and Community

The code repository will contain detailed instructions for using the CLI tool

Milestone Summary

Milestone No.	Milestone Summary & Staffing	Funding	Estimated Timeframe
1	Design Test Vector && Export Test Vectors From Ethereum & 2 people	$16800	2 weeks
2	FEVM consumes Test Vector and generates test report & 2 people	$25200	3 weeks

Total Budget Requested

$42000

Maintenance and Upgrade Plans

We are willing to maintain this test vector tool for a long time.

Team

Contact Info

wenxin@froghub.io

Team Members

• He Zheng zhenghe@froghub.io
• Minghang Fan （slack）
• OM - FrogHub （slack）
• Plus-FrogHub （slack）

Team Website

https://www.froghub.io/

Relevant Experience

Please describe (in words) your team's relevant experience, and why you think would do a great job with this RFP. You can cite your team's prior experience in similar domains, doing similar dev work, individual team members' backgrounds, etc.

1. We are familiar with the implementation of EVM. By using the source code of the public contract, we can output the StateChanges caused by Transaction on Ethereum in a human-readable form.
2. We are familiar with the implementation of FVM/FEVM, and understand the underlying data structure (eg. HAMT AMT BitField).
3. We are familiar with the implementation of tvx and the design of its Test Vector.

Team code repositories

https://github.com/froghub-io

[BlocksOnAChain]

CC: @maciejwitowski for visibility

[FroghubMan]

https://github.com/froghub-io/fevm-test-vectors/blob/main/corpus/example/0x406076cf80ee9bb77393ac5a07dd5611bce80b80b95b78bc75be93869bb7dac7.json

[FroghubMan]

builtin-actors adds test_tvx. The test vectors data of evm is implemented and directly written into the vm. Finally, the output results are compared with the changed values of the slots. There is a bug in the rlp_append implementation of RlpCreateAddress. issues

[raulk]

Hi @FroghubMan, thanks for your proposal. Although your description and explanations are quite brief, it seems like you have a good big picture understanding of the approach involved. Some comments here:

• Have you considered using the tvx vector format instead of an entirely new format? This would save you considerable effort since you can drop the "FEVM consumes vector" task since we already have a conformance test vector runner covering ref-fvm only. In the future, we could also reuse the Lotus runner.
• Doing the above will allow you to hit ref-fvm instead of the EVM actor through a mock runtime, thus providing for broader testing coverage (very desirable).
• How does your method deal with cross-contract calls? You would need to detect during extraction (EVM tracing) and instantiate the right actors with the right addresses in the state tree.
• How does your method deal with environmental data, e.g. blockhash (tipset CID)? We will need to extend the tvx vector schema to hold that data so it can be returned through the externs.
• How does your method deal with calls to opcodes that access the state tree, e.g. BALANCE, EXTCODE*, etc.?

[FroghubMan]

@raulk At the beginning, we considered tvx vector format, but because the comparison target of tvx vector format is StateTree Root, which is inconsistent with the EVM slot we imagined, so we wanted to design a new format test vector.

For environmental data, we need to expand the implementation of opcodes related to it, implement the function that these environment data can be read from external data sources in testing phase. These data sources can be files, blockstore, etc.

For opcodes that access the state tree, e.g. BALANCE, EXTCODE*,We can use the address, balance, , bytecode(if needed) and slot data(if needed) on the state tree to correctly instantiate the account. for opcode EXTCODEHASH, BLOCKHASH, since it is generated by cid, it is necessary to read directly from the outside (maybe file/blockstore as datasource) in testing phase.

Now after our team's discussion, we can indeed directly use the tvx vector format and reuse the conformance test vector runner. We intend to achieve our goals as follows:

1. Intercept the data of the following types of opcodes through evm tracing: context: BLOCKHASH, COINBASE, GASPRICE, GAS, TIMESTAMP, NUMBER, DIFFICULTY, GASLIMIT, CHAINID, BASEFEE ext: EXTCODESIZE, EXTCODECOPY, EXTCODEHASH call: CALLDATALOAD, CALLDATASIZE, CALLDATACOPY, CODESIZE, CODECOPY, CALL storage: SLOAD, SSTORE

2. With the help of ref-fvm, correctly simulate the data (address, balance, bytecode, slot data) of the account on the state tree through the opcode data of call and storage types,In order to execute the contract call.

3. Expand the function implementation of context and ext type opcodes under ref-fvm, allowing data to be read from external data sources (eg, files, blockstore, etc.).

4. Expand the tvx function so that the test vector of the evm type can be replayed

[wenxinnnnn]

To summarize the previous discussion, I submitted a new issue: #1202