search

ethereum / execution-spec-tests

A Python framework and collection of test cases to generate test vectors for Ethereum execution clients
https://ethereum.github.io/execution-spec-tests
MIT License
91 stars 63 forks source link

EOF Tracker #500

Open marioevz opened 3 months ago

marioevz commented 3 months ago 
### Tasks
- [x] Create a test spec type for the EOF container fixture format ([Done](https://github.com/ethereum/execution-spec-tests/commit/0022b4d94e3c91513e1b73090732f6e3b348953b)) 
- [ ] Update exceptions.py to add the existing EOF exceptions to the [Enum](https://github.com/ethereum/execution-spec-tests/blob/760a67e03baa23d782c9c8762c13ca6089920bda/src/ethereum_test_tools/exceptions/exceptions.py#L167)
- [ ] Add automatic BlockchainTest and BlockchainHiveTest generation to the [`EOFTest`](https://github.com/ethereum/execution-spec-tests/blob/760a67e03baa23d782c9c8762c13ca6089920bda/src/ethereum_test_tools/spec/eof/eof_test.py#L16) spec.
- [ ] Update or create a new class to be able to generate and parametrize EOF containers according to the latest spec
- [ ] Port [EIP3540](https://github.com/ethereum/tests/tree/develop/src/EOFTestsFiller/EIP3540)
- [ ] Port [EIP3670](https://github.com/ethereum/tests/tree/develop/src/EOFTestsFiller/EIP3670)
- [ ] Port [EIP4200](https://github.com/ethereum/tests/tree/develop/src/EOFTestsFiller/EIP4200)
- [ ] Port [EIP4750](https://github.com/ethereum/tests/tree/develop/src/EOFTestsFiller/EIP4750)
- [ ] Port [EIP5450](https://github.com/ethereum/tests/tree/develop/src/EOFTestsFiller/EIP5450)
marioevz commented 3 months ago

Basic example of the test filler: https://github.com/ethereum/execution-spec-tests/blob/760a67e03baa23d782c9c8762c13ca6089920bda/tests/prague/eip3540_evm_object_format_v1/test_eof.py#L19

This is not how all tests should be rewritten as this example barely takes any advantage of parameterization, but it's simply an example of usage of the EOFTest spec class.

gumb0 commented 3 months ago

To summarize the current status of our existing tests:

  1. https://github.com/ethereum/tests/tree/develop/src/EIPTestsFiller/StateTests/stEOF/stEIP3540 https://github.com/ethereum/tests/tree/develop/src/EIPTestsFiller/StateTests/stEOF/stEIP4200 https://github.com/ethereum/tests/tree/develop/src/EIPTestsFiller/StateTests/stEOF/stEIP4750 https://github.com/ethereum/tests/tree/develop/src/EIPTestsFiller/StateTests/stEOF/stEIP5450

    • These tests were written a while ago for Shanghai version of the spec, after that they were mostly only updated to keep them compatible with the later changes. We removed all tests testing EOF validation aspect and made validation tests from them (see below). I wouldn't expect these tests to cover all complexities of current version of EIPs-3540, 4200, 4750, 5450 (but they do pass with current spec)

  2. EOF validation tests https://github.com/ethereum/tests/tree/develop/src/EOFTestsFiller

    • These are partially hand-written (EIP* folders, also efExample and ori), and partially generated from evmone unit tests, which @winsvega merged recently as Copier tests (rest of ef* folders). I expect these tests to cover almost all aspects of validation reasonably well.

  3. https://github.com/ethereum/tests/pull/1324

    • State tests generated from evmone tests, these mostly test various aspects of contract creation (EOFCREATE, TXCREATE, InitcodeTransactions), new EXTCALL* instructions and some related execution aspects.

See also the list at https://github.com/ipsilon/eof/blob/main/spec/implementation_matrix.md#column-descriptions we try to keep it up to date.