ethereum / execution-spec-tests

A Python framework and collection of test cases to generate test vectors for Ethereum execution clients
https://ethereum.github.io/execution-spec-tests
MIT License
106 stars 70 forks source link

print an error if restricted opcodes detected in state_test function #899

Open winsvega opened 2 days ago

winsvega commented 2 days ago

I found the issue of at least one of the cases the eels resover reports an error

.venv/lib/python3.12/site-packages/requests/models.py:974: in json
    return complexjson.loads(self.text, **kwargs)
/usr/lib/python3.12/json/__init__.py:346: in loads
    return _default_decoder.decode(s)
/usr/lib/python3.12/json/decoder.py:337: in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
/usr/lib/python3.12/json/decoder.py:355: in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
E   json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:
tests/frontier/scenarios/test_scenarios.py:257: in test_scenarios
    state_test(env=tx_env, pre=pre, post=post, tx=tx, post_hint=hint)
src/pytest_plugins/filler/filler.py:836: in __init__
    fixture = self.generate(
src/ethereum_test_specs/state.py:188: in generate
    return self.make_state_test_fixture(t8n, fork, eips, self.post_hint)
src/ethereum_test_specs/state.py:139: in make_state_test_fixture
    transition_tool_output = t8n.evaluate(
src/ethereum_clis/transition_tool.py:487: in evaluate
    return self._evaluate_server(t8n_data=t8n_data, debug_output_path=debug_output_path)
src/ethereum_clis/transition_tool.py:314: in _evaluate_server
    output: TransitionToolOutput = TransitionToolOutput.model_validate(response.json())
.venv/lib/python3.12/site-packages/requests/models.py:978: in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
E   requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
------------------------------------------------------------------------- Captured stderr call --------------------------------------------------------------------------
----------------------------------------
Exception occurred during processing of request from ['local', 0]
Traceback (most recent call last):
  File "/usr/lib/python3.12/socketserver.py", line 318, in _handle_request_noblock
    self.process_request(request, client_address)
  File "/usr/lib/python3.12/socketserver.py", line 349, in process_request
    self.finish_request(request, client_address)
  File "/home/wins/.cache/ethereum-spec-evm-resolver/Cancun/src/ethereum_spec_tools/evm_tools/daemon.py", line 91, in finish_request
    super().finish_request(request, client_address)
  File "/usr/lib/python3.12/socketserver.py", line 362, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python3.12/socketserver.py", line 761, in __init__
    self.handle()
  File "/usr/lib/python3.12/http/server.py", line 436, in handle
    self.handle_one_request()
  File "/usr/lib/python3.12/http/server.py", line 424, in handle_one_request
    method()
  File "/home/wins/.cache/ethereum-spec-evm-resolver/Cancun/src/ethereum_spec_tools/evm_tools/daemon.py", line 66, in do_POST
    main(args=args, out_file=out_wrapper, in_file=input)
  File "/home/wins/.cache/ethereum-spec-evm-resolver/Cancun/src/ethereum_spec_tools/evm_tools/__init__.py", line 109, in main
    return t8n_tool.run()
           ^^^^^^^^^^^^^^

so if a state test uses blockhash opcode it causes pydantic to panic and eels resolver makes issues in the logs. which is actually we should report an error. if a state test mode uses blockhash opcode it should be a blockchain test as we agreed to remove fake hash caluclations from t8n

winsvega commented 2 days ago

to reproduce

make a test with this bytecode

def test_blockhash_bug(state_test: StateTestFiller, pre: Alloc):
    """
    comment
    """
    contract = pre.deploy_contract(code=Op.BLOCKHASH(0))
    tx = Transaction(
        sender=pre.fund_eoa(),
        gas_limit=500_000_000,
        gas_price=10,
        to=contract,
        data=b"",
        value=0,
        protected=False,
    )

    state_test(env=Environment(), pre=pre, post={}, tx=tx)

here is the evmdump http://retesteth.ethdevops.io/temp/pyspecs/frontier__scenarios__test_scenarios__test_blockhash_bug/