ethereum / execution-spec-tests

A Python framework and collection of test cases to generate test vectors for Ethereum execution clients
https://ethereum.github.io/execution-spec-tests
MIT License
105 stars 72 forks source link

feat(fw,forks,tests): Add EVM code type marker #610

Closed marioevz closed 1 month ago

marioevz commented 3 months ago

🗒️ Description

@pytest.mark.with_all_evm_code_types marker

Adds the with_all_evm_code_types marker that can be added to a test as follows:

@pytest.mark.with_all_evm_code_types
@pytest.mark.valid_from("Frontier")
def test_something_with_all_evm_code_types(evm_code_type: EVMCodeType):
    pass

and the test will be parameterized for parameter evm_code_type only with value [EVMCodeType.LEGACY] starting on fork Frontier, and eventually it will be parametrized with with values [EVMCodeType.LEGACY, EVMCodeType.EOF_V1] on the EOF activation fork.

Adding the evm_code_type parameter to the function signature is optional and can be done only if required.

In all calls to pre.deploy_contract, if the code parameter is Bytecode type, and evm_code_type==EVMCodeType.EOF_V1, the bytecode will be automatically wrapped in an EOF V1 container.

Code wrapping might fail in the following circumstances:

In the case where the code wrapping fails, evm_code_type can be added as a parameter to the test and the bytecode can be dynamically modified to be compatible with the EOF V1 container.

@pytest.mark.with_all_call_opcodes marker

This marker is used to automatically parameterize a test with all EVM call opcodes that are valid for the fork being tested.

import pytest

@pytest.mark.with_all_call_opcodes
@pytest.mark.valid_from("Frontier")
def test_something_with_all_call_opcodes(pre: Alloc, call_opcode: Op):
    ...

In this example, the test will be parametrized for parameter call_opcode with values [Op.CALL, Op.CALLCODE] starting on fork Frontier, [Op.CALL, Op.CALLCODE, Op.DELEGATECALL] on fork Homestead, [Op.CALL, Op.CALLCODE, Op.DELEGATECALL, Op.STATICCALL] on fork Byzantium, and eventually it will be parametrized with with values [Op.CALL, Op.CALLCODE, Op.DELEGATECALL, Op.STATICCALL, Op.EXTCALL, Op.EXTSTATICCALL, Op.EXTDELEGATECALL] on the EOF activation fork.

Parameter evm_code_type will also be parametrized with the correct EVM code type for the opcode under test.

Code Generators Switch and Conditional

Code generators Switch and Conditional have been updated to take the evm_code_type and correctly generate EOF V1 code in case of EVMCodeType.EOF_V1.

--evm-code-type parameter for fill

Parameter --evm-code-type to force all tests to be filled with an specific EVM code type and, while most tests would currently fail because of their incompatibility with EOF, it can serve the purpose of exploring which tests could be easily updated for EOF.

call_return_code helper

This function returns the correct return code depending on the give opcode, whether a success is expected or not, and whether a revert is expected or not.

It is useful to automatically set the correct storage expectations depending on the type of call used, since all legacy *CALL opcodes and the EOF EXT*CALL opcodes return different values.

Fork Covariant Processor Changes

Changes were required to src/pytest_plugins/forks/forks.py to allow markers to parametrize two or more parameters at the same time, depending on the fork. It was necessary because the new marker with_all_call_opcodes parametrizes call_opcode and evm_code_type at the same time.

Example tests updated

The following tests were updated as an example of how an updated test would look like:

🔗 Related Issues

None

✅ Checklist

marioevz commented 1 month ago

Should we instead create a single "with all" marker:

* `pytest.mark.with_all(...)`

That accepts a list of potential input/s:

* `pytest.mark.with_all("tx_types", "evm_code_types")`

This is a really nice idea, although it would require some refactoring to the covariant fork parameter logic, so I think this could be its own PR. Also would need to make some modifications because, as it is right now, with_all_evm_code_types and with_all_call_opcodes are mutually exclusive.