muellerberndt / laser-ethereum

Symbolic virtual machine for Ethereum
MIT License
68 stars 20 forks source link

Adding tests #6

Closed freewind closed 6 years ago

freewind commented 6 years ago

Note: There are some logic in svm.py is not covered, since I really can't construct correct test cases for them without deep understanding of Ethereum VM. So sorry~

muellerberndt commented 6 years ago

Hi @freewind, tests return the following on my High Sierra system:

$ python --version
Python 3.6.3

$ pip freeze
asn1crypto==0.24.0
BTrees==4.4.1
certifi==2018.1.18
cffi==1.11.5
chardet==3.0.4
coincurve==7.1.0
cytoolz==0.9.0.1
dictionaries==0.0.1
eth-hash==0.1.1
eth-utils==1.0.1
ethereum==2.3.1
future==0.16.0
idna==2.6
laser-ethereum==0.5.19
mythril==0.15.7
pbkdf2==1.3
persistent==4.2.4.2
py-ecc==1.4.2
py-flags==1.1.2
py-solc==2.1.0
pycparser==2.18
pycryptodome==3.5.1
pyethash==0.1.27
pysha3==1.0.2
PyYAML==3.12
repoze.lru==0.7
requests==2.18.4
rlp==0.6.0
scrypt==0.8.6
semantic-version==2.6.0
six==1.11.0
toolz==0.9.0
transaction==2.2.1
urllib3==1.22
z3-solver==4.5.1.0.post2
zc.lockfile==1.2.1
ZConfig==3.2.0
ZODB==5.4.0
zodbpickle==1.0
zope.interface==4.4.3

$ ./all_tests.sh 
F..F...FF..s
======================================================================
FAIL: test_with_file_calls (tests.svm_test.SvmTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/bernhardmueller/Projects/laser-ethereum/tests/svm_test.py", line 83, in test_with_file_calls
    self._test_with_file("calls.sol")
  File "/Users/bernhardmueller/Projects/laser-ethereum/tests/svm_test.py", line 80, in _test_with_file
    self.assertEqual(generated_info, expected_info, "{}: information of laser is changed".format(str(input_file)))
AssertionError: '{\n [18542 chars]0x029f"\n                },\n                {[1856681 chars]2\n}' != '{\n [18542 chars]0x029b"\n                },\n                {[1817567 chars]2\n}' : /Users/bernhardmueller/Projects/laser-ethereum/tests/testdata/inputs/calls.sol: information of laser is changed

======================================================================
FAIL: test_with_file_kinds_of_calls (tests.svm_test.SvmTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/bernhardmueller/Projects/laser-ethereum/tests/svm_test.py", line 92, in test_with_file_kinds_of_calls
    self._test_with_file("kinds_of_calls.sol")
  File "/Users/bernhardmueller/Projects/laser-ethereum/tests/svm_test.py", line 80, in _test_with_file
    self.assertEqual(generated_info, expected_info, "{}: information of laser is changed".format(str(input_file)))
AssertionError: '{\n [11916 chars]0x0277"\n                },\n                {[2252708 chars]2\n}' != '{\n [11916 chars]0x0273"\n                },\n                {[2214350 chars]2\n}' : /Users/bernhardmueller/Projects/laser-ethereum/tests/testdata/inputs/kinds_of_calls.sol: information of laser is changed

======================================================================
FAIL: test_with_file_returnvalue (tests.svm_test.SvmTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/bernhardmueller/Projects/laser-ethereum/tests/svm_test.py", line 104, in test_with_file_returnvalue
    self._test_with_file("returnvalue.sol")
  File "/Users/bernhardmueller/Projects/laser-ethereum/tests/svm_test.py", line 80, in _test_with_file
    self.assertEqual(generated_info, expected_info, "{}: information of laser is changed".format(str(input_file)))
AssertionError: '{\n [7195 chars]0x00d8"\n                },\n                {[587078 chars]2\n}' != '{\n [7195 chars]0x00d4"\n                },\n                {[567622 chars]2\n}' : /Users/bernhardmueller/Projects/laser-ethereum/tests/testdata/inputs/returnvalue.sol: information of laser is changed

======================================================================
FAIL: test_with_file_rubixi (tests.svm_test.SvmTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/bernhardmueller/Projects/laser-ethereum/tests/svm_test.py", line 107, in test_with_file_rubixi
    self._test_with_file("rubixi.sol")
  File "/Users/bernhardmueller/Projects/laser-ethereum/tests/svm_test.py", line 80, in _test_with_file
    self.assertEqual(generated_info, expected_info, "{}: information of laser is changed".format(str(input_file)))
AssertionError: '{\n [272909 chars]058200ac57644d4"\n                }\n         [33198915 chars]2\n}' != '{\n [272909 chars]05820cdd8b03056"\n                }\n         [33198915 chars]2\n}' : /Users/bernhardmueller/Projects/laser-ethereum/tests/testdata/inputs/rubixi.sol: information of laser is changed

----------------------------------------------------------------------
Ran 12 tests in 41.192s

FAILED (failures=4, skipped=1)
muellerberndt commented 6 years ago

Result of eth.coinbase in get is "etherbase must be explicitly specified" (as I haven't set an etherbase in geth). I'd say just remove all those IPC and RPC call tests, we're never going to use them anyway!

freewind commented 6 years ago

@b-mueller I found current test case report is not readable when output is different, I'm trying to make a better solution, wait a minute

freewind commented 6 years ago

@b-mueller Done, please see my new PR: https://github.com/b-mueller/laser-ethereum/pull/7

freewind commented 6 years ago

@b-mueller

Result of eth.coinbase in get is "etherbase must be explicitly specified" (as I haven't set an etherbase in geth). I'd say just remove all those IPC and RPC call tests, we're never going to use them anyway!

I prefer that too, it's meaningless to testing the unused methods!

I also think you may involve to the topcoder forums to give such important information.

These tests spent me a whole day and makes not much sense.

Coding without communication with client directly is not agile

muellerberndt commented 6 years ago

I agree, real-time communication would be better. I'll look into joining the TopCoder forums next time.

freewind commented 6 years ago

@b-mueller I'm still working on making the tests better, and will also remove some unused ipc/rpc methods. Will create some PRs later