evm: wrong result from blockhash operation

[holiman]

A failing state-test. Here's the last line of agreement, and the differing line:

-------
prev:           both: {"pc":79,"op":64,"gas":"0x79bc38","gasCost":"0x0","memSize":0,"stack":["0x0","0x1","0x1","0x2","0x2","0xffff","0x1f4","0x78859e5b97166c486532b1595a673e9f9073643f1b519c6f18511b9913","0x2","0x389","0x0","0x0","0x1","0x0","0x3e3d6d5ff042148d326c1898713a76759ca273","0x0"],"depth":1,"refund":0,"opName":"BLOCKHASH"}
diff:         geth-0: {"pc":80,"op":65,"gas":"0x79bc24","gasCost":"0x0","memSize":0,"stack":["0x0","0x1","0x1","0x2","0x2","0xffff","0x1f4","0x78859e5b97166c486532b1595a673e9f9073643f1b519c6f18511b9913","0x2","0x389","0x0","0x0","0x1","0x0","0x3e3d6d5ff042148d326c1898713a76759ca273","0x44852b2a670ade5407e78fb2863c51de9fcb96542a07186fe3aeda6bb8a116d"],"depth":1,"refund":0,"opName":"COINBASE"}
diff:    besubatch-0: {"pc":80,"op":65,"gas":"0x79bc24","gasCost":"0x0","memSize":0,"stack":["0x0","0x1","0x1","0x2","0x2","0xffff","0x1f4","0x78859e5b97166c486532b1595a673e9f9073643f1b519c6f18511b9913","0x2","0x389","0x0","0x0","0x1","0x0","0x3e3d6d5ff042148d326c1898713a76759ca273","0x0"],"depth":1,"refund":0,"opName":"COINBASE"}

It's a BLOCKHASH with stack being [... , "0x0"]. On the next step, geth has the top stack value 0x44852b2a670ade5407e78fb2863c51de9fcb96542a07186fe3aeda6bb8a116d, besu has 0x0.

We are executing on block 1:

martin@mediaNUK:~$ cat /tmp/00000936-mixed-1.json | jq .   | grep currentNumber
      "currentNumber": "0x1",

This means that block 0 is eligible. The hash 0x4485... is crypto.Keccak256([]byte{'0'}) (in geth common.BytesToHash(crypto.Keccak256([]byte(big.NewInt(int64(n)).String())))). For any valid block number (valid as in within the range of allowed lookups), the state tests use that formula. (For evm t8n, the rules are stricter: it should error out if a blockhash op is invoked and the necessary blockhashes are not supplied by the caller).

Nethermind:

martin@mediaNUK:~$ /home/martin/workspace/nethtest --trace -m -i /tmp/00000936-mixed-1.json 2>&1 | grep COINBASE
{"pc":80,"op":65,"gas":"0x79bc24","gasCost":"0x2","memSize":0,"stack":["0x0","0x1","0x1","0x2","0x2","0xffff","0x1f4","0x78859e5b97166c486532b1595a673e9f9073643f1b519c6f18511b9913","0x2","0x389","0x0","0x0","0x1","0x0","0x3e3d6d5ff042148d326c1898713a76759ca273","0x44852b2a670ade5407e78fb2863c51de9fcb96542a07186fe3aeda6bb8a116d"],"depth":1,"refund":0,"opname":"COINBASE","error":""}

Geth:

martin@mediaNUK:~$ /home/martin/workspace/evm --json --noreturndata --nomemory statetest /tmp/00000936-mixed-1.json 2>&1 | grep COINBASE
{"pc":80,"op":65,"gas":"0x79bc24","gasCost":"0x2","memSize":0,"stack":["0x0","0x1","0x1","0x2","0x2","0xffff","0x1f4","0x78859e5b97166c486532b1595a673e9f9073643f1b519c6f18511b9913","0x2","0x389","0x0","0x0","0x1","0x0","0x3e3d6d5ff042148d326c1898713a76759ca273","0x44852b2a670ade5407e78fb2863c51de9fcb96542a07186fe3aeda6bb8a116d"],"depth":1,"refund":0,"opName":"COINBASE"}

Nimbus-eth1:

martin@mediaNUK:~$ /home/martin/workspace/nimbus-eth1/tools/evmstate/evmstate --json --noreturndata --nomemory --nostorage /tmp/00000936-mixed-1.json 2>&1 | grep COINBASE
{"pc":80,"op":65,"gas":"0x79bc24","gasCost":"0x2","memSize":0,"opName":"COINBASE","depth":1,"stack":["0x0","0x1","0x1","0x2","0x2","0xffff","0x1f4","0x78859e5b97166c486532b1595a673e9f9073643f1b519c6f18511b9913","0x2","0x389","0x0","0x0","0x1","0x0","0x3e3d6d5ff042148d326c1898713a76759ca273","0x44852b2a670ade5407e78fb2863c51de9fcb96542a07186fe3aeda6bb8a116d"]}

Besu:

martin@mediaNUK:~$ /home/martin/workspace/besu-vm --json state-test /tmp/00000936-mixed-1.json 2>&1 | grep COINBASE
{"pc":80,"op":65,"gas":"0x79bc24","gasCost":"0x2","memSize":0,"stack":["0x0","0x1","0x1","0x2","0x2","0xffff","0x1f4","0x78859e5b97166c486532b1595a673e9f9073643f1b519c6f18511b9913","0x2","0x389","0x0","0x0","0x1","0x0","0x3e3d6d5ff042148d326c1898713a76759ca273","0x0"],"depth":1,"refund":0,"opName":"COINBASE"}

testcase:

{"00000936-mixed-1":{"env":{"currentCoinbase":"b94f5374fce5edbc8e2a8697c15331677e6ebf0b","currentDifficulty":"0x20000","currentRandom":"0x0000000000000000000000000000000000000000000000000000000000020000","currentGasLimit":"0x26e1f476fe1e22","currentNumber":"0x1","currentTimestamp":"0x3e8","previousHash":"0x0000000000000000000000000000000000000000000000000000000000000000","currentBaseFee":"0x10"},"pre":{"0x00000000000000000000000000000000000000f1":{"code":"0x600060016001600260026101f461ffff5817907c78859e5b97166c486532b1595a673e9f9073643f1b519c6f18511b991383610389895f8a5b82723e3d6d5ff042148d326c1898713a76759ca27383404148083e7788908f409e38766cfafe083134530b5279797ef56394faa3183b3780475932387ea37a326494ff1b6b57018c79f07a9ba492a256543f0b6a813b316835b15385077d9e8d32f18c3bb159fab11734a363999e7e940a9c19699da08a82a3ff6743b13f44206194321d026490411a938b35f47cf31b91a1ff397b7c6d879f119915176f3699801bf0766b62fd855b60607d6f133f35fa40307e150a096ea35041306b9330f23441473018055f44698e928a109b02f38f515386177e819161487a5bfa7951f5317183fa743d7587f479009c8b4813089d8d3cf37e3a9b6170528b84f296f07433363a766a64529d8c62ff859c0852173e875685ff15123594f13418966f483a999990109b1694511007477171068aa03a561b8df4743642452039a43c391c3e0033f4615ab1f37cb10293387613024673187c1d5b731c3e53a19080b03e6b94755580876f7a6bf0f238f0991c775172893e7b14449e8c050968b032a29bf13265657d11386d9f8c6f360670fa1116178909f59a369c7c005f8d1464527365f0fd5b818499a2469a0b173f42478294f11c8b40567d8b9f72f43594028d67b17893668a999d8d205014687a6a9b09198bfe87a144313895a208183c8b4060a392387a5106f235a019f36d336f6034f5fe006df413a090791c3c6c52b181f047457a8a73f294f401467a79048060113b1b20fd87078a78a273368c116741128c9a715b085312949c15460b9b8613587c79365b643a67106d840b51a0009e3f893f893236527d540b14403a866d6a147df465a03a8a62489a1d7f680962036b716111f45572367e887606f282820b3d7c178af5f3846a60f2999290f3f3847882f19362113e668f19a07bfffa8aa4374793a28401fe34fd93551a7a721256130789707767a408847307067a85fe3cf1f58a64f0368d38413299f083941885867d18308e0835b054531793087b8c970490597203978d1a8a1b6ba459621b8a8d8313655711563e31073e3c3d6056337299089a94415468097df1425a91f209b13d77f493425041f4ff6718441445463c519771069e727ef59214b19b0b5a3a3b3aa01b6c963da1638117323d637c92443e97795a438593155f73123417fd7b01397b7f4587b1727a011a12450903fa62f182b168920b945a57f28a2078665088593458137d75729f8898800953933056867c1d48441839128c6238f39d3c01149b9859b1958e0b806c81966302f03407a0f5047d778c52018b20f5f13e5b7255544632708064318b31fd0b7b420b3659761da06a17584554453d4363a27c8f767941619d628b75934051","storage":{"0x0000000000000000000000000000000000000000000000000000000000000000":"0x000000000000000000000000000000000000000000000000000000000000000a","0x0000000000000000000000000000000000000000000000000000000000000002":"0x000000000000000000000000000000000000000000000000000000000000000a","0x0000000000000000000000000000000000000000000000000000000000000003":"0x0000000000000000000000000000000000000000000000000000000000000002","0x0000000000000000000000000000000000000000000000000000000000000005":"0x0000000000000000000000000000000000000000000000000000000000000007","0x0000000000000000000000000000000000000000000000000000000000000008":"0x0000000000000000000000000000000000000000000000000000000000000011","0x0000000000000000000000000000000000000000000000000000000000000009":"0x0000000000000000000000000000000000000000000000000000000000000005","0x000000000000000000000000000000000000000000000000000000000000000c":"0x0000000000000000000000000000000000000000000000000000000000000008"},"balance":"0x0","nonce":"0x0"},"0xa94f5374fce5edbc8e2a8697c15331677e6ebf0b":{"code":"0x","storage":{},"balance":"0xffffffffff","nonce":"0x0"}},"transaction":{"gasPrice":"0x10","nonce":"0x0","to":"0x00000000000000000000000000000000000000f1","data":["0xfe25cf37540066ffd30066af7690fad6d555e13067f5c3c16ced98a803e2b009db4ab89b243ac17b80386d12d621723c363e46aff1fcce9c084d"],"gasLimit":["0x7a1200"],"value":["0xdbbe"],"secretKey":"0x45a915e4d060149eb4365960e6a7a45f334393093061116b197e3240065ff2d8"},"out":"0x","post":{"Shanghai":[{"hash":"0x0000000000000000000000000000000000000000000000000000000000000000","logs":"0x0000000000000000000000000000000000000000000000000000000000000000","indexes":{"data":0,"gas":0,"value":0}}]}}}

[holiman]

Older version of besu, at commit 51956b7f3 does not have this error:

$ /home/martin/workspace/besu-vm --json state-test /tmp/00000936-mixed-1.json 2>&1 | grep COINBASE
{"pc":80,"op":65,"gas":"0x79bc24","gasCost":"0x2","memory":"0x","memSize":0,"stack":["0x0","0x1","0x1","0x2","0x2","0xffff","0x1f4","0x78859e5b97166c486532b1595a673e9f9073643f1b519c6f18511b9913","0x2","0x389","0x0","0x0","0x1","0x0","0x3e3d6d5ff042148d326c1898713a76759ca273","0x44852b2a670ade5407e78fb2863c51de9fcb96542a07186fe3aeda6bb8a116d"],"returnData":"0x","depth":1,"refund":0,"opName":"COINBASE","error":""}

[shemnon]

It's the test env, and I argue that Besu's interpretation, while differing from the past is correct, although I accept in the interim it may need to be rolled back for test conformance.

The test env in the reference test is as follows (reformatted for clarity)...

      "env": {
        "currentCoinbase": "b94f5374fce5edbc8e2a8697c15331677e6ebf0b",
        "currentDifficulty": "0x20000",
        "currentRandom": "0x0000000000000000000000000000000000000000000000000000000000020000",
        "currentGasLimit": "0x26e1f476fe1e22",
        "currentNumber": "0x1",
        "currentTimestamp": "0x3e8",
        "previousHash": "0x0000000000000000000000000000000000000000000000000000000000000000",
        "currentBaseFee": "0x10"
      },

The previous hash is Zero, the current block is 1, ergo the hash of block zero is zero, per the env setup. However, the reference test envs expect blockhash to be the the keccak of the uint256 representation of the block number in question.

If the "parentHash" field is removed, the test performs as expected.

[holiman]

Oh, wow, yeah the previousHash, I hadn't thought of that. Good point. It also means I can work around this by supplying a previousHash of 0x44852b2a670ade5407e78fb2863c51de9fcb96542a07186fe3aeda6bb8a116d. Or, as you say, drop it.

[shemnon]

So... bug or no bug? I have a patch ready to roll.

[holiman]

It is a bug, in my opinion (but my fuzzing can work around it so I don't personally care if you fix it or not)

[shemnon]

The scope of this is strictly the state-test subcommand of the evmtool. All state tests run by the state test harness worked and all production networks use a different block-hash rule than the fuzzing frameworks.

[holiman]

Thinking some more about this: I would not say that besu is 'wrong' or has a 'bug'. The instructions are unclear: what is even the point of previousHash if we just ignore it?

But apparently all other clients behave differently, and I guess that would be the only reason to change the behaviour. As I said, for my fuzzing, it doesn't matter, I just want to avoid tripping over false positives.

Feel free to close this regardless of whether you merge #5126 or not