Error: Val is a not a valid unsigned 256-word on some contracts

cdump commented 4 days ago

Hey, great project!

I've added it to my EVMole benchmarks here and found that there are errors in some contracts.

To find contracts with errors and reproduce:

$ git clone --recurse-submodules https://github.com/cdump/evmole
$ cd evmole/benchmarks
$ PROVIDERS_SELECTORS=sevm make sevm.build sevm.selectors/largest1k
$ python3 compare.py --datasets largest1k --providers etherscan sevm --mode selectors --show-errors

...
dataset largest1k (1000 contracts, 24427 signatures), sevm:
  time: 34.1s
  False Positive: 0 signatures, 0 contracts
  False Negative: 492 signatures, 16 contracts
  errors:
    3451E5418eE5A925F3e786DFC673Ba0B0fBE5936
...

$ docker run --rm -v `pwd`/datasets:/datasets --entrypoint node -it evmole-bench-sevm

> new (await import("sevm")).Contract(JSON.parse((await import('fs')).readFileSync(`/datasets/largest1k/0x3451E5418eE5A925F3e786DFC673Ba0B0fBE5936.json`))['code'])
Uncaught:
Error: Val is a not a valid unsigned 256-word: 124330809102446660538845562036705210025114037699336929360115994223289874178937783844864
    at new Val (/app/node_modules/sevm/dist/src/ast/index.js:117:19)
    at Shl.eval (/app/node_modules/sevm/dist/src/ast/alu.js:274:47)
    at Sar.eval (/app/node_modules/sevm/dist/src/ast/alu.js:296:32)
    at Local.eval (/app/node_modules/sevm/dist/src/ast/index.js:140:66)
    at DIVEXPsig (/app/node_modules/sevm/dist/src/step.js:239:29)
    at ALU.EQ (/app/node_modules/sevm/dist/src/step.js:261:19)
    at ConstantinopleStep.EQ (/app/node_modules/sevm/dist/src/step.js:731:31)
    at EVM.exec (/app/node_modules/sevm/dist/src/evm.js:123:40)
    at EVM.run (/app/node_modules/sevm/dist/src/evm.js:92:18)

You can also see the random50k and vyper datasets in the same way.

P.S. If you know a better way to get function selectors (or maybe arguments too, but without using an external 4bytes database), feel free to contact me. Currently, I'm using it this way: https://github.com/cdump/evmole/blob/master/benchmark/providers/sevm/main.mjs#L21

acuarica commented 3 days ago

Hey @cdump thanks for sending this, I'll take a look when I have some time. Glad to see that you have included it in your benchmark 🙏🏼.

If you know a better way to get function selectors (or maybe arguments too, but without using an external 4bytes database), feel free to contact me. Currently, I'm using it this way: https://github.com/cdump/evmole/blob/master/benchmark/providers/sevm/main.mjs#L21

That's perfect, this doesn't use any external 4byte database. See for example https://github.com/acuarica/evm/blob/8b58da73a6364e870b0be0f2f452e69d8989f3b2/bin/sevm.mjs#L247-L252

The 4byte external lookup is decoupled from the main library through the patch method https://acuarica.github.io/evm/tsdoc/classes/index.Contract.html#patch.

Regarding the selectors detection, this is what sevm roughly does https://acuarica.github.io/evm/#/internals?id=detection-of-function-selectors. Hope it helps.

cdump commented 3 days ago

> new (await import("sevm")).Contract('0x60016101111b5ffd')

Error: Val is a not a valid unsigned 256-word: 15177100720513508366558296147058741458143803430094840009779784451085189728165691392
    at new Val (/app/node_modules/sevm/dist/src/ast/index.js:117:19)
    at Shl.eval (/app/node_modules/sevm/dist/src/ast/alu.js:274:47)
...

[00]    PUSH1   01
[02]    PUSH2   0111
[05]    SHL 
[06]    PUSH0   
[07]    REVERT

doc: value << shift: the shifted value. If shift is bigger than 255, returns 0.

acuarica commented 2 days ago

Thanks for finding this example. It should be fixed in the latest release https://github.com/acuarica/evm/releases/tag/v0.6.18.

cdump commented 2 days ago

Looks like fixed - https://github.com/cdump/evmole/commit/e59004f48a320955c65828825a26ebd75dd21392 But there are still some other errors in random50k & vyper datatests, you can find it yourself by following my steps from the 1st post in this issue

acuarica / evm

Error: Val is a not a valid unsigned 256-word on some contracts #125