search

ethereum / retesteth

testeth via RPC. Test run, generation by t8ntool protocol
http://retesteth.ethdevops.io/
GNU General Public License v3.0
113 stars 75 forks source link

Retesteth usage in a private network #212

Open Infra1515 opened 1 year ago

Infra1515 commented 1 year ago

Hello, I am trying to run retesteth against a custom client based on pre-merge geth v1.11.* that is utilising PoW consensus inside a private network. I am writing because I feel that are some knowledge gaps of how to use it and hoping to get some help. Excuse me in advance if this is not the correct place. Just for reference, I am running on the code on arm (M1 Pro).

Here are the steps that I am doing:

  1. Since I experienced various problem with docker the retest code is compiled locally using the instructions from https://github.com/ethereum/retesteth/blob/develop/circle.yml.
  2. Downloaded the test repo - https://github.com/ethereum/tests.
  3. I copied the evm binary from myCustomClient/build/bin/evm to $GOPATH/evm and am able to successfully run tests with the following command
  4. Started my node on localhost:8545
  5. Execute the test with the following command ./retesteth/build/retesteth/retesteth -t GeneralStateTests -- --testpath /tests --nodes 0.0.0.0:8545

So far so good, the tests run and 99% of them pass. What I am having trouble understanding is the following:

No errors detected WARNING: Test folder /Users/.myClient/end-to-end/tests/src/BlockchainTestsFiller/InvalidBlocks/bcForgedTest appears to be unused! Total Tests Run: 0



Not sure If I am using retesteth in the correct manner for testing a private network. Any help or materials/guidance will be appreciated. 
Thank you in advance :)
giskook commented 1 year ago

Can retesteth run test through RPC?

winsvega commented 1 year ago

sorry for the late response. message me on @wdimitry telegram I help with the setup.

yes, test RPC are still supported. but if you use fork of geth you must use t8n. if you develop your own client you can develop RPC if this is preferable for you. RPC is not supported by pyspecs so you wont be able to generate .py tests. (but able to run it)

to use nodes the config must be TCP socket style. not t8n. when you create custom config you must run it with --clients customconfig to use it. otherwise it will use default config all subfolders in .retesteth/default folder are used by any other configs unless explicitly defined so to use custom genesis .json you need to define it in .retesteth/customconfig/genesis subfolder.

zjg555543 commented 1 week ago

Any

sorry for the late response. message me on @wdimitry telegram I help with the setup.

yes, test RPC are still supported. but if you use fork of geth you must use t8n. if you develop your own client you can develop RPC if this is preferable for you. RPC is not supported by pyspecs so you wont be able to generate .py tests. (but able to run it)

to use nodes the config must be TCP socket style. not t8n. when you create custom config you must run it with --clients customconfig to use it. otherwise it will use default config all subfolders in .retesteth/default folder are used by any other configs unless explicitly defined so to use custom genesis .json you need to define it in .retesteth/customconfig/genesis subfolder.

Hey guy. Thanks for your sharing. I got the similar issue by runing on X Layer private network. Could you help to check it? image

/usr/bin/retesteth -t GeneralStateTests/stSystemOperationsTest -- --datadir /tests/config --clients defaultout --testpath /tests 
Running tests using path: /tests
Running 1 test case...
Retesteth config path: /tests/config
Active client configurations: 'defaultout '
Running tests for config 'Ethereum GO on TCP' 2
Test Case "stSystemOperationsTest": (1 of 1)
Instantiated: "result":"cdk-erigon/2.0-upstream/v2.0.0-beta18-bde1b5ecf931c9b367a7ebc0b1d60c0830f83493/linux-amd64/go1.20.10"
WARNING: , Message: The command '/tests/config/defaultout/start.sh 1 2>/dev/null' exited with 512 code., has empty debugInfo! Missing debug Testinfo for test step.
terminate called after throwing an instance of 'test::EthError'
  what():  The command '/tests/config/defaultout/start.sh 1 2>/dev/null' exited with 512 code.
Finishing retesteth run

*** 1 failure is detected in the test module "EthereumTests"
unknown location(0): fatal error: in "GeneralStateTests/stSystemOperationsTest": signal: SIGSEGV, si_code: 128 (memory access violation at address: 0x00000000)
/retesteth/retesteth/testSuites/statetests/StateTestsBoost.cpp(102): last checkpoint: "stSystemOperationsTest" fixture ctor
*** Total Tests Run: 1

*** stack smashing detected ***: terminated