(core-geth:EVMCv10) evmone fails GeneralStateTests/stRandom/randomStatetest153.json

Context

Core-Geth has supported EVMCv7 for a while, and now we're upgrading (finally) to EVMCv10.

PR: https://github.com/etclabscore/core-geth/pull/507 (branch: https://github.com/etclabscore/core-geth/tree/evmc-v10)
Development branch: https://github.com/etclabscore/core-geth/tree/evmc-v10-develop

Problem

To test core-geth's support of the EVMC host interface, we run the cross-client state tests with a configured external interpreter.

go test -v ./tests -run State -vm.evm=path/to/libevmone.so

:x: When we run these tests with evmone, we see a great many test failures.
:heavy_check_mark: When we run these test with hera, we see no failures; everything passes.
:heavy_check_mark: When we run these test without an external interpreter, we see no failures; everything passes.
The problem seems relevant for nearly all fork configurations.

We've taken stRandom/randomStatetest153.json as an example, and have created a minimal test focused on debugging the problem. This test is in the bespoke development branch at evmc/evmc_test.go, and can be run with:

go test -v ./evmc

This test/program runs the state test against the native, hera, and evmone interpreters and compares the results, comparing state dumps in case of mismatches.

The problem in our isolated test case is that evmone appears to fail to assign storage properly (via SSTORE), resulting in a mismatched storage state (and balance!) and, of course, state root hashes. It passes a zero-value value to the host interface method SetStorage instead of 0x20000.

The test result, comparing a natively-executed state dump (left), with the state from the evmone execution (right):

The contract:

echo "44420b4442432055" | disasm
44420b4442432055
0      DIFFICULTY
1      TIMESTAMP
2      SIGNEXTEND
3      DIFFICULTY
4      TIMESTAMP
5      NUMBER
6      SHA3
7      SSTORE

Questions

Are we in the right place? Does this smell like an evmc issue? Or an evmone issue?
How do we get the traces from evmone?

Notes

The EVMC flag is also available in other application contexts provided by the repo.

build/bin/geth --vm.evm=path/to/libevm.so
build/bin/evm --vm.evm=path/to/libevm.so statetest
build/bin/evm --vm.evm=path/to/libevm.so t8n

Core-Geth has some install scripts that can be convenient for installing and testing external interpreters.

make evmone
make hera
make test-evmc

Reproduce issue

# Get client codebase at development version
git clone --branch evmc-v10-develop https://github.com/etclabscore/core-geth.git
cd core-geth
git submodule update --init --recursive # clone x-client tests subdirs

# Get external EVMs
make hera
make evmone

# Run an isolated test for debugging
go test -v ./evmc

# Run the test suite with hera and evmone
make test-evmc

cc @ziogaschr

I figured this one out. It was the PrevRandao field in the TxContext that was mismatched (visible by inspecting the stack after the DIFFICULTY operation). This field has been, in Ethereum, with EIP-1559, renamed and repurposed from Difficulty to represent the VM's context random value. This patch fixes this issue.

Rationale

EVMOne trace output via -vm.evm=path/to/libemvone.so,trace=on, which writes a trace to stdout.

{"depth":0,"rev":"Homestead","static":false}
{"pc":0,"op":68,"opName":"DIFFICULTY","gas":378932,"stack":[],"memorySize":0}
{"pc":1,"op":66,"opName":"TIMESTAMP","gas":378930,"stack":["0x0"],"memorySize":0}
{"pc":2,"op":11,"opName":"SIGNEXTEND","gas":378928,"stack":["0x0","0x3e8"],"memorySize":0}
{"pc":3,"op":68,"opName":"DIFFICULTY","gas":378923,"stack":["0x0"],"memorySize":0}
{"pc":4,"op":66,"opName":"TIMESTAMP","gas":378921,"stack":["0x0","0x0"],"memorySize":0}
{"pc":5,"op":67,"opName":"NUMBER","gas":378919,"stack":["0x0","0x0","0x3e8"],"memorySize":0}
{"pc":6,"op":32,"opName":"KECCAK256","gas":378917,"stack":["0x0","0x0","0x3e8","0x1"],"memorySize":0}
{"pc":7,"op":85,"opName":"SSTORE","gas":378597,"stack":["0x0","0x0","0xae72e2bf2302ebcd309e003e5be58830f96deddaf87bb89eeea159388bfe3ec1"],"memorySize":1024}
{"error":null,"gas":373597,"gasUsed":5335,"output":""}

Middle; a comparison of EVMOne and native traces.

- TIMESTAMP = 0x3e8 / 0x3e8
- DIFFICULTY = 0x0 / 0x20000
- NUMBER = 0x1 / 0x1

Native trace.

{"pc":0,"op":68,"gas":"0x5c834","gasCost":"0x2","memSize":0,"stack":[],"depth":1,"refund":0,"opName":"DIFFICULTY"}
{"pc":1,"op":66,"gas":"0x5c832","gasCost":"0x2","memSize":0,"stack":["0x20000"],"depth":1,"refund":0,"opName":"TIMESTAMP"}
{"pc":2,"op":11,"gas":"0x5c830","gasCost":"0x5","memSize":0,"stack":["0x20000","0x3e8"],"depth":1,"refund":0,"opName":"SIGNEXTEND"}
{"pc":3,"op":68,"gas":"0x5c82b","gasCost":"0x2","memSize":0,"stack":["0x20000"],"depth":1,"refund":0,"opName":"DIFFICULTY"}
{"pc":4,"op":66,"gas":"0x5c829","gasCost":"0x2","memSize":0,"stack":["0x20000","0x20000"],"depth":1,"refund":0,"opName":"TIMESTAMP"}
{"pc":5,"op":67,"gas":"0x5c827","gasCost":"0x2","memSize":0,"stack":["0x20000","0x20000","0x3e8"],"depth":1,"refund":0,"opName":"NUMBER"}
{"pc":6,"op":32,"gas":"0x5c825","gasCost":"0x140","memSize":0,"stack":["0x20000","0x20000","0x3e8","0x1"],"depth":1,"refund":0,"opName":"KECCAK256"}
{"pc":7,"op":85,"gas":"0x5c6e5","gasCost":"0x4e20","memory":"0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","memSize":1024,"stack":["0x20000","0x20000","0xae72e2bf2302ebcd309e003e5be58830f96deddaf87bb89eeea159388bfe3ec1"],"depth":1,"refund":0,"opName":"SSTORE"}
{"pc":8,"op":0,"gas":"0x578c5","gasCost":"0x0","memory":"0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000","memSize":1024,"stack":["0x20000"],"depth":1,"refund":0,"opName":"STOP"}
{"output":"","gasUsed":"0x4f6f","time":135695}

We discovered and fixed, then, a subsequent and unrelated issue with the way core-geth implemented the EIP2200 SetStorage method around the comparison of *uint256.Int types; https://github.com/etclabscore/core-geth/commit/724fa5be47bb772db23fa4a9a1a3bbe6e65cdd9e

ethereum / evmc