search

trailofbits / manticore

Symbolic execution tool
https://blog.trailofbits.com/2017/04/27/manticore-symbolic-execution-for-humans/
GNU Affero General Public License v3.0
3.69k stars 472 forks source link

AttributeError: Can't pickle local object 'ManticoreEVM.finalize.<locals>.worker_finalize' #1715

Closed anhdungle93 closed 4 years ago

anhdungle93 commented 4 years ago

Summary of the problem

I tried to use manticore (installed in a conda environment) on the following contract

pragma solidity >=0.4.24 <0.6.0;
contract Simple {
    function f(uint a) payable public{
        if (a == 65) {
            revert();
        }
    }
}

with two manticore versions, one directly from pip and one from git master branch. The one from pip has the error:

2020-05-24 13:29:34,985: [30082] m.c.manticore:WARNING: Manticore is only supported on Linux. Proceed at your own risk!
2020-05-24 13:29:35,248: [30082] m.main:INFO: Registered plugins: DetectReentrancySimple, DetectEnvInstruction, DetectUnusedRetVal, DetectSuicidal, DetectExternalCallAndLeak, <class 'manticore.ethereum.plugins.KeepOnlyIfStorageChanges'>, DetectUninitializedMemory, DetectInvalid, DetectReentrancyAdvanced, DetectManipulableBalance, DetectIntegerOverflow, DetectUninitializedStorage, DetectDelegatecall
2020-05-24 13:29:35,249: [30082] m.main:INFO: Beginning analysis
2020-05-24 13:29:35,251: [30082] m.e.manticore:INFO: Starting symbolic create contract
2020-05-24 13:29:36,619: [30082] m.e.manticore:INFO: Starting symbolic transaction: 0
2020-05-24 13:29:36,899: [30082] m.e.manticore:INFO: 1 alive states, 3 terminated states
2020-05-24 13:29:36,964: [30082] m.e.manticore:INFO: Starting symbolic transaction: 1
2020-05-24 13:29:37,326: [30082] m.e.manticore:INFO: 1 alive states, 6 terminated states
Traceback (most recent call last):
  File "/Users/anhdungle/opt/anaconda3/envs/manticore_env/bin/manticore", line 8, in <module>
    sys.exit(main())
  File "/Users/anhdungle/opt/anaconda3/envs/manticore_env/lib/python3.8/site-packages/manticore/__main__.py", line 42, in main
    ethereum_main(args, logger)
  File "/Users/anhdungle/opt/anaconda3/envs/manticore_env/lib/python3.8/site-packages/manticore/ethereum/cli.py", line 123, in ethereum_main
    m.finalize(only_alive_states=args.only_alive_testcases)
  File "/Users/anhdungle/opt/anaconda3/envs/manticore_env/lib/python3.8/site-packages/manticore/core/manticore.py", line 128, in newFunction
    return func(self, *args, **kw)
  File "/Users/anhdungle/opt/anaconda3/envs/manticore_env/lib/python3.8/site-packages/manticore/ethereum/manticore.py", line 1645, in finalize
    proc.start()
  File "/Users/anhdungle/opt/anaconda3/envs/manticore_env/lib/python3.8/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/Users/anhdungle/opt/anaconda3/envs/manticore_env/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/Users/anhdungle/opt/anaconda3/envs/manticore_env/lib/python3.8/multiprocessing/context.py", line 283, in _Popen
    return Popen(process_obj)
  File "/Users/anhdungle/opt/anaconda3/envs/manticore_env/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/Users/anhdungle/opt/anaconda3/envs/manticore_env/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/Users/anhdungle/opt/anaconda3/envs/manticore_env/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/Users/anhdungle/opt/anaconda3/envs/manticore_env/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'ManticoreEVM.finalize.<locals>.worker_finalize'

and the one from git master

2020-05-24 13:30:20,881: [30469] m.c.manticore:WARNING: Manticore is only supported on Linux. Proceed at your own risk!
2020-05-24 13:30:21,194: [30469] m.main:INFO: Registered plugins: DetectManipulableBalance, DetectInvalid, DetectUnusedRetVal, DetectUninitializedStorage, DetectExternalCallAndLeak, DetectDelegatecall, DetectReentrancySimple, DetectIntegerOverflow, DetectReentrancyAdvanced, DetectEnvInstruction, DetectSuicidal, DetectUninitializedMemory
2020-05-24 13:30:21,194: [30469] m.main:INFO: Beginning analysis
2020-05-24 13:30:21,196: [30469] m.e.manticore:INFO: Starting symbolic create contract
2020-05-24 13:30:22,717: [30469] m.e.manticore:INFO: Starting symbolic transaction: 0
2020-05-24 13:30:22,999: [30469] m.e.manticore:INFO: 1 alive states, 5 terminated states
2020-05-24 13:30:23,059: [30469] m.e.manticore:INFO: Starting symbolic transaction: 1
2020-05-24 13:30:23,398: [30469] m.e.manticore:INFO: 1 alive states, 8 terminated states
Traceback (most recent call last):
  File "/Users/anhdungle/opt/anaconda3/envs/manticore2_env/bin/manticore", line 11, in <module>
    load_entry_point('manticore', 'console_scripts', 'manticore')()
  File "/Users/anhdungle/learning/manticore/manticore/__main__.py", line 45, in main
    ethereum_main(args, logger)
  File "/Users/anhdungle/learning/manticore/manticore/ethereum/cli.py", line 145, in ethereum_main
    m.finalize(only_alive_states=args.only_alive_testcases)
  File "/Users/anhdungle/learning/manticore/manticore/core/manticore.py", line 175, in newFunction
    return func(self, *args, **kw)
  File "/Users/anhdungle/learning/manticore/manticore/ethereum/manticore.py", line 1715, in finalize
    proc.start()
  File "/Users/anhdungle/opt/anaconda3/envs/manticore2_env/lib/python3.8/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/Users/anhdungle/opt/anaconda3/envs/manticore2_env/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/Users/anhdungle/opt/anaconda3/envs/manticore2_env/lib/python3.8/multiprocessing/context.py", line 283, in _Popen
    return Popen(process_obj)
  File "/Users/anhdungle/opt/anaconda3/envs/manticore2_env/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/Users/anhdungle/opt/anaconda3/envs/manticore2_env/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/Users/anhdungle/opt/anaconda3/envs/manticore2_env/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/Users/anhdungle/opt/anaconda3/envs/manticore2_env/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'ManticoreEVM.finalize.<locals>.worker_finalize'

Manticore version

both are of version 0.3.3

Python version

the python version of the environment for the version from pip: 3.8.2 the python version of the environment for the version from git master: 3.8.3

OS / Environment

MacOS Catalina

ehennenfent commented 4 years ago

Your example exhibits the above error on OSX using Python3.8, but works just fine with Python3.7. Looking at the traceback, it seems as though Python3.8 uses the fork syscall for pickling, which doesn't work very well on OSX. Our MacOS branch uses threads in order to avoid this, but that doesn't seem to have carried over to the pickler. I'll need to do some more digging to see whether this is something we can fix or if we need to wait for upstream changes to multiprocessing.

Edit: Clarified that threads are independent from multiprocessing

anhdungle93 commented 4 years ago

@ehennenfent Thanks a lot, I switched to Python 3.7.6, but got a different error 

manticore SafeMath.sol 
2020-05-27 09:17:31,174: [902] m.c.manticore:WARNING: Manticore is only supported on Linux. Proceed at your own risk!
2020-05-27 09:17:31,832: [902] m.main:INFO: Registered plugins: DetectIntegerOverflow, DetectSuicidal, <class 'manticore.ethereum.plugins.KeepOnlyIfStorageChanges'>, DetectDelegatecall, DetectUninitializedStorage, DetectExternalCallAndLeak, DetectUninitializedMemory, DetectInvalid, DetectEnvInstruction, DetectReentrancySimple, DetectManipulableBalance, DetectReentrancyAdvanced, DetectUnusedRetVal
2020-05-27 09:17:31,832: [902] m.main:INFO: Beginning analysis
2020-05-27 09:17:31,835: [902] m.e.manticore:INFO: Starting symbolic create contract
2020-05-27 09:17:35,065: [902] m.e.manticore:INFO: Starting symbolic transaction: 0
2020-05-27 09:24:12,065: [902] m.e.manticore:INFO: 0 alive states, 15 terminated states
2020-05-27 09:24:12,290: [902] m.e.manticore:INFO: Starting symbolic transaction: 1
2020-05-27 09:24:12,396: [1113] m.c.manticore:INFO: Generated testcase No. 0 - RETURN(1 txs)
2020-05-27 09:24:12,400: [1115] m.c.manticore:INFO: Generated testcase No. 2 - REVERT(2 txs)
2020-05-27 09:24:12,407: [1119] m.c.manticore:INFO: Generated testcase No. 3 - REVERT(2 txs)
2020-05-27 09:24:12,407: [1120] m.c.manticore:INFO: Generated testcase No. 4 - REVERT(2 txs)
2020-05-27 09:24:12,408: [1114] m.c.manticore:INFO: Generated testcase No. 1 - REVERT(2 txs)
2020-05-27 09:24:12,410: [1122] m.c.manticore:INFO: Generated testcase No. 5 - REVERT(2 txs)
2020-05-27 09:24:12,418: [1121] m.c.manticore:INFO: Generated testcase No. 6 - RETURN(2 txs)
2020-05-27 09:24:12,450: [1118] m.c.manticore:INFO: Generated testcase No. 7 - REVERT(2 txs)
2020-05-27 09:24:12,454: [1117] m.c.manticore:INFO: Generated testcase No. 8 - RETURN(2 txs)
2020-05-27 09:24:12,503: [1116] m.c.manticore:INFO: Generated testcase No. 9 - REVERT(2 txs)
2020-05-27 09:24:13,600: [1113] m.c.manticore:INFO: Generated testcase No. 10 - REVERT(2 txs)
2020-05-27 09:24:13,893: [1116] m.c.manticore:INFO: Generated testcase No. 11 - RETURN(2 txs)
2020-05-27 09:24:13,899: [1122] m.c.manticore:INFO: Generated testcase No. 12 - RETURN(2 txs)
2020-05-27 09:24:24,000: [1115] m.c.manticore:INFO: Generated testcase No. 13 - RETURN(2 txs)
2020-05-27 09:24:37,905: [1114] m.c.manticore:INFO: Generated testcase No. 14 - REVERT(2 txs)
Process Process-2:
Traceback (most recent call last):
  File "/Users/anhdungle/opt/anaconda3/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/Users/anhdungle/opt/anaconda3/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/anhdungle/opt/anaconda3/lib/python3.7/site-packages/manticore/ethereum/manticore.py", line 1633, in worker_finalize
    finalizer(q.get_nowait())
  File "/Users/anhdungle/opt/anaconda3/lib/python3.7/site-packages/manticore/ethereum/manticore.py", line 1628, in finalizer
    self.generate_testcase(st, message=message)
  File "/Users/anhdungle/opt/anaconda3/lib/python3.7/site-packages/manticore/ethereum/manticore.py", line 1515, in generate_testcase
    is_something_symbolic = state.platform.dump(stream, state, self, message)
  File "/Users/anhdungle/opt/anaconda3/lib/python3.7/site-packages/manticore/platforms/evm.py", line 3075, in dump
    balance = state.solve_one(balance)
  File "/Users/anhdungle/opt/anaconda3/lib/python3.7/site-packages/manticore/core/state.py", line 340, in solve_one
    return self.solve_one_n(expr, constrain=constrain)[0]
  File "/Users/anhdungle/opt/anaconda3/lib/python3.7/site-packages/manticore/core/state.py", line 358, in solve_one_n
    value = self._solver.get_value(self._constraints, expr)
  File "/Users/anhdungle/opt/anaconda3/lib/python3.7/site-packages/manticore/core/smtlib/solver.py", line 632, in get_value
    raise SolverError("Model is not available")
manticore.exceptions.SolverError: Model is not available
2020-05-27 09:31:54,351: [902] m.c.manticore:INFO: Results in /Users/anhdungle/learning/smart_contracts_test/manticore_tests/mcore_bp5jbhp2
anhdungle93 commented 4 years ago

I switched to python 3.6 and everything seems fine now.

ehennenfent commented 4 years ago

Sorry, didn't see your last reply come in! For future reference, the "Model not available" error means that Z3 was unable to solve for a value that Manticore asked it for. That could be because some of the constraints placed on it were contradictory, or potentially due to available disk space or memory limitations. If you come across a piece of code you're able to share that works properly on Python 3.6 but fails on later versions, please do share it. Discrepancies between major versions of Python are always something we're concerned about. Thanks for filing!

trojanMcAfee commented 2 years ago

I was getting this error, switched to python 3.6.1 through a conda environment and now I'm getting AttributeError: 'NoneType' object has no attribute 'result'. Any ideas how to solve this?

Full error:

Traceback (most recent call last):
  File "/Users/cdgmachado/anaconda3/envs/myenv/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/Users/cdgmachado/anaconda3/envs/myenv/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/cdgmachado/anaconda3/envs/myenv/lib/python3.6/site-packages/manticore/ethereum/manticore.py", line 1756, in worker_finalize
    finalizer(q.get_nowait())
  File "/Users/cdgmachado/anaconda3/envs/myenv/lib/python3.6/site-packages/manticore/ethereum/manticore.py", line 1747, in finalizer
    if only_alive_states and last_tx.result in {"REVERT", "THROW", "TXERROR"}:
AttributeError: 'NoneType' object has no attribute 'result'

This is how I'm calling manticore, with remaps included: manticore contracts/ethereum/ProxyFactory.sol --solc-remaps "@openzeppelin=node_modules/@openzeppelin @uniswap=node_modules/@uniswap @chainlink=node_modules/@chainlink @rari-capital=node_modules/@rari-capital hardhat=node_modules/hardhat" --contract ProxyFactory.sol