test(evm-ci): Run Ethereum protocol tests across the chain in CI/CD

[Unique-Divine]

A security team that specializes in auditing Ethereum and EVM-compatible Layer 1 blockchains has made a recommendation. They suggest implementing an automated test suite that would check for consistent behavior between Nibiru and Ethereum. This aims to ensure that Nibiru's behavior aligns with Ethereum's in key areas.

Recommended Tools:

• https://github.com/ethereum/tests
• https://github.com/ethereum/retesteth

I think the entry point to start the ticket is here where it mentions testing a modified geth in the docs.

[onikonychev]

Running retesteth

First, grab the docker with tests runner and a running script:

cd ~

wget http://retesteth.ethdevops.io/dretesteth.tar

docker image load --input dretesteth.tar

wget https://raw.githubusercontent.com/ethereum/retesteth/master/dretesteth.sh
chmod +x dretesteth.sh

Run the default config for the first time to have ~/tests/config folder produced.

--datadir /tests/config path is within docker - it will be created inside ~/tests.

./dretesteth.sh -t GeneralStateTests/stExample -- \
 --testpath ~/tests --datadir /tests/config

Create folder nibiru in the ~/tests/config dir. Put there this attached file with the name config.

Run nibiru localnet:

make localnet

Run the tests with config nibiru:

./dretesteth.sh -t GeneralStateTests/stExample -- \
 --testpath ~/tests --datadir /tests/config --clients nibiru

It'll output the error because test_setChainParams rpc method is not implemented.

Error: RPCImpl::test_setChainParamsNoGenesis is not implemented! (GeneralStateTests/stExample/add11, step: test_setChainParams: Cancun)

--------
TestOutputHelper detected 1 errors during test execution!
/retesteth/retesteth/helpers/TestOutputHelper.cpp(240): error: in "GeneralStateTests/stExample":

[onikonychev]

RPC methods required for retesteth

https://github.com/ethereum/retesteth/wiki/RPC-Methods

[onikonychev]

Looks like the requirements for the RPC methods are not actual and don't actually work. I implemented the first required endpoint test_setChainParams but was still getting this error. Then I opened the source code or retesteth and found that this endpoint is "Not Implemented" (link to the code):

void RPCImpl::test_setChainParamsNoGenesis(spSetChainParamsArgs const& _config)
{
    (void) _config;
    ETH_FAIL_MESSAGE("RPCImpl::test_setChainParamsNoGenesis is not implemented!");
}

Next, I tried to execute at least one test which does not do test_setChainParams. The only one I found was TransactionTests suite.

./dretesteth.sh -t TransactionTests -- --all \
 --testpath ~/tests --datadir /tests/config --clients nibiru

That test failed because it required test_rawTransaction RPC method but that method not even documented. I dummied it too following the call parameters but it requires some value inresult which format is unknown. Without any doc and just reading the source code it appears to be really hard.

[onikonychev]

Also found a suite EOFTests but it required another non-documented call test_rawEOFCode.