Some tests can be executed alone with the evmbin (like statetests) but others not (like blockchaintests)
There's no way to "fill" tests. At this moment the only way to do it is by implementing the
retesteth tool and this will introduce a lot of complexity in the current client. Not being able to fill tests means having less capacity to do experiments to test some functionality when
things go wrong.
the proposal is to extend the current openthereum-evm tool to
Run the ethereum/tests from evmbin. This functionality will be removed from the ethcore.
Do not embeed the tests, and run them from a ethereum/test folder
Move json testing code and test-chainspecs to the evmbin
Having a json file with instructions about how to run an ethereum/test, this is done partially now with the current.json file, but there is not a definition about what to do with some of them (ignore some, known issues, tag stress tests & legacy tests). This also allows having different testing scenarios.
Start implementing state transition json tests https://github.com/ethereum/go-ethereum/pull/20958. It is still in draft, but it seems that it could be implemented quickly.
Having a filler with support for yul/solidity code for state transition.
At this moment, the current way the tests are managed by openetherum could be improved:
evmbin(like statetests) but others not (like blockchaintests)the proposal is to extend the current openthereum-evm tool to
https://github.com/ethereum/go-ethereum/pull/20958. It is still in draft, but it seems that it could be implemented quickly.