search

ethereum / execution-spec-tests

A Python framework and collection of test cases to generate test vectors for Ethereum execution clients
https://ethereum.github.io/execution-spec-tests
MIT License
118 stars 74 forks source link

✨(gentest): make `gentest` great #866

Open danceratopz opened 1 month ago

danceratopz commented 1 month ago

The gentest CLI is a really nice tool that can generate and write a Python (pytest) test module to disk given a type 0 transaction hash on a live network. This generates a state_test from the mainnet transaction 0xa41f...06be:

uv run gentest 0xa41f343be7a150b740e5c939fa4d89f3a2850dbe21715df96b612fc20d1906be tests/paris/test_0xa41f.py

Currently, due to its limited transaction type support (type 0 only), it's of limited use. This meta ticket is to track progress of making gentest an easy-to-use CLI tool that can generate test cases for specific issues observed on devnets, and potentially, to allow benchmarking of an EVM's transaction processing using blockchain_tests created from real mainnet data.

While working on the issues below, we should consider and try to ensure that gentest also works on Verkle and EOF testnets.

### `gentest` Tasks
- [ ] https://github.com/ethereum/execution-spec-tests/issues/893
- [ ] #891
- [ ] #863
- [ ] #864
- [ ] #859
- [ ] #860
- [ ] #861
- [ ] #862
- [ ] #865
- [ ] https://github.com/ethereum/execution-spec-tests/issues/908
### `testinit` Tasks
- [ ] #926 
- [ ] #927
- [ ] https://github.com/ethereum/execution-spec-tests/issues/945
- [ ] https://github.com/ethereum/execution-spec-tests/issues/946
danceratopz commented 1 month ago

@raxhvl said he's happy to pick this up! :relaxed:

Thanks @raxhvl! Above I tried to outline the most obvious tasks for gentest. Please see these as suggestions - if you have any other ideas or inputs, just go for it! I'm also happy to jump in and help out if need be.

raxhvl commented 1 month ago

Thanks @danceratopz I will comment on individual issues as I start working on them.