Update BadOpcode fillers for Cancun

[enitrat]

The createAllBadOpcodeTests.sh was not updated to generate tests for the Ex and Fx families for the Cancun fork. Therefore, they were not being tested with the --fork Cancun parameter.

I haven't been able to generate the test files from the fillers, as I couldn't set retesteth with Docker properly.

[winsvega]

but those are invalid opcodes as they are not implemented in cancun no ?

[winsvega]

@enitrat can you generate the affected tests?

[enitrat]

@winsvega I couldn't get retesteth to work with Docker to generate the tests, I will try again a bit later

[winsvega]

I think it need a new golang in the dockerfile to build geth

[qbzzt]

but those are invalid opcodes as they are not implemented in cancun no ?

Yes, my bad - I thought they were implemented in Cancun, but I was wrong. Sorry I haven't been keeping in touch - life's crazy.

[enitrat]

I think it need a new golang in the dockerfile to build geth

What's the easiest way to generate the tests? Should I build retesteth from source?

[winsvega]

the easies way is to use docker. since it auto setup retesteth, geth and other clients. and then you just run

./dretesteth.sh build
./dretesteth.sh install
# from the test repository
dr testFiller.json --filltests 

make sure to use retesteth::develop branch can edit Dockerfile to build only geth and skip other clients.

[winsvega]

after docker is complete, run

cd src/GeneralStateTestsFiller
dr stBadOpcode --filltests --filloutdated 
dr stBadOpcode --filltests --fillchain --filloutdated 

[enitrat]

I'm still getting an issue with Docker:

❯ ./dretesteth.sh install

./dretesteth.sh: line 7: -3: substring expression < 0
/usr/bin/retesteth -t ///tests -- install --testpath --clients t8ntool
Error: `--testpath` option is missing an argument.

edit: it seems that this might be a problem with how the extension of the file is being removed. after fixing this by changing SCRIPT_NAME_LINK="${SCRIPT_NAME::-3}" to SCRIPT_NAME_LINK="${SCRIPT_NAME%.*}", another issue:

❯ ./dretesteth.sh install
ln: /usr/bin/dretesteth: Operation not permitted
Added link /usr/bin/dretesteth >> /Users/msaug/deps/retesteth/dretesteth.sh
ln: /usr/bin/dr: Operation not permitted
Added link /usr/bin/dr >> /Users/msaug/deps/retesteth/dretesteth.sh

I modified the dockerfile to only keep ARG GETH="master" and ARG RETESTETH="develop"

edit2: OS X prevents changes to /usr/bin, the symbolic link should be made to /usr/local/bin instead

[winsvega]

I see this issues are from how mac runs the .sh script

Feel free to modify it. I then test that it still works under linux.

Did it work after changes? The install command is not a requirement you can run the script directly. Also set the --testpath variable.

retesteth.sh -t GeneralStateTests/stBadOpcodes -- --testpath /data/tests --filltests --filloutdated

[enitrat]

I'm still having an issue where Can't find geth's 'evm' executable alias in the system path!

Partial trace:

❯ dr stBadOpcode --filltests --filloutdated

/usr/bin/retesteth -t GeneralStateTests/stBadOpcode -- --filltests --filloutdated --testpath /tests --clients t8ntool
Running tests using path: /tests
Running 1 test case...
Options path `/root/.retesteth` doesn't exist, attempt to create a new directory
Retesteth config path: /root/.retesteth
Active client configurations: 't8ntool '
Running tests for config 'Ethereum GO on StateTool' 2
Test Case "stBadOpcode": (1 of 1)
Can't find geth's 'evm' executable alias in the system path!
WARNING: Reading empty result for /root/.retesteth/t8ntool/start.sh -v
WARNING: replaceCode returned empty bytecode `0x` trying to compile  (GeneralStateTests/stBadOpcode/opcEADiffPlaces, step: GeneralStateTestFiller)

...

Can't find geth's 'evm' executable alias in the system path!
Finishing retesteth run
/retesteth/retesteth/helpers/TestOutputHelper.cpp(240): error: in "GeneralStateTests/stBadOpcode":
*** Total Tests Run: 0

Error: The command '/root/.retesteth/t8ntool/start.sh -v' exited with 256 code. (GeneralStateTests/stBadOpcode/, step: TestSuite::executeTest)
Error: The command '/root/.retesteth/t8ntool/start.sh --state.fork Cancun --state.reward 0 --state.chainid 1 --input.alloc /dev/shm/0c9f9ae1-71c4-403d-97ea-7a49dd060c66/alloc.json --input.txs /dev/shm/0c9f9ae1-71c4-403d-97ea-7a49dd060c66/txs.rlp --input.env /dev/shm/0c9f9ae1-71c4-403d-97ea-7a49dd060c66/env.json --output.basedir /dev/shm/0c9f9ae1-71c4-403d-97ea-7a49dd060c66 --output.result out.json --output.alloc outAlloc.json --output.errorlog /dev/shm/0c9f9ae1-71c4-403d-97ea-7a49dd060c66/error.json' exited with 256 code. (GeneralStateTests/stBadOpcode/opcEADiffPlaces, fork: Cancun, TrInfo: d: 0, g: 0, v: 0, TrData: `:label normal :abi f(uint) 0x00..`)
Error: ERROR OCCURED FILLING TESTS: The command '/root/.retesteth/t8ntool/start.sh --state.fork Cancun --state.reward 0 --state.chainid 1 --input.alloc /dev/shm/0c9f9ae1-71c4-403d-97ea-7a49dd060c66/alloc.json --input.txs /dev/shm/0c9f9ae1-71c4-403d-97ea-7a49dd060c66/txs.rlp --input.env /dev/shm/0c9f9ae1-71c4-403d-97ea-7a49dd060c66/env.json --output.basedir /dev/shm/0c9f9ae1-71c4-403d-97ea-7a49dd060c66 --output.result out.json --output.alloc outAlloc.json --output.errorlog /dev/shm/0c9f9ae1-71c4-403d-97ea-7a49dd060c66/error.json' exited with 256 code. (GeneralStateTests/stBadOpcode/opcEADiffPlaces, fork: Cancun, TrInfo: d: 0, g: 0, v: 0, TrData: `:label normal :abi f(uint) 0x00..`)

[winsvega]

Docker failed to build geth. Check that the go version is 1.21 inside dockerfile. Or the logs. Sorry that it doesn't work, I can do it from my system, but it is good to have your feedback to fix all this issues

The latest docker build shows green. There is retesteth.sh rebuild command to disable cache.

[enitrat]

Indeed it looks like the geth build failed with

# runtime/cgo
gcc: error: unrecognized command-line option '-m64'
Geth is empty

[winsvega]

https://github.com/ethereum/tests/pull/1362