search

angr / claripy

An abstraction layer for constraint solvers.
BSD 2-Clause "Simplified" License
286 stars 94 forks source link

bug: RecursionError: maximum recursion depth exceeded #111

Closed winter2020 closed 5 years ago

winter2020 commented 5 years ago

The test code:

import angr
import claripy

def check(binary_path):
    proj = angr.Project(binary_path, use_sim_procedures=True, default_analysis_mode='symbolic_approximating', auto_load_libs=False)
    cfg = proj.analyses.CFGEmulated(keep_state=True, enable_advanced_backward_slicing=True, context_sensitivity_level=4)
    vfg = proj.analyses.VFG(cfg, function_start=proj.entry, context_sensitivity_level=2, interfunction_level=4, remove_options={angr.options.OPTIMIZE_IR})

if __name__ == '__main__':
    test_program = './test'
    check(test_program)

Error:

  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/claripy-8.18.10.25-py3.5.egg/claripy/ast/base.py", line 887, in ite_excavated
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/claripy-8.18.10.25-py3.5.egg/claripy/ast/base.py", line 831, in _excavate_ite
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/claripy-8.18.10.25-py3.5.egg/claripy/ast/base.py", line 831, in <listcomp>
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/claripy-8.18.10.25-py3.5.egg/claripy/ast/base.py", line 887, in ite_excavated
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/claripy-8.18.10.25-py3.5.egg/claripy/ast/base.py", line 831, in _excavate_ite
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/claripy-8.18.10.25-py3.5.egg/claripy/ast/base.py", line 831, in <listcomp>
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/claripy-8.18.10.25-py3.5.egg/claripy/ast/base.py", line 887, in ite_excavated
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/claripy-8.18.10.25-py3.5.egg/claripy/ast/base.py", line 831, in _excavate_ite
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/claripy-8.18.10.25-py3.5.egg/claripy/ast/base.py", line 831, in <listcomp>
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/claripy-8.18.10.25-py3.5.egg/claripy/ast/base.py", line 887, in ite_excavated
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/claripy-8.18.10.25-py3.5.egg/claripy/ast/base.py", line 831, in _excavate_ite
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/claripy-8.18.10.25-py3.5.egg/claripy/ast/base.py", line 831, in <listcomp>
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/claripy-8.18.10.25-py3.5.egg/claripy/ast/base.py", line 887, in ite_excavated
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/claripy-8.18.10.25-py3.5.egg/claripy/ast/base.py", line 831, in _excavate_ite
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/claripy-8.18.10.25-py3.5.egg/claripy/ast/base.py", line 831, in <listcomp>
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/claripy-8.18.10.25-py3.5.egg/claripy/ast/base.py", line 887, in ite_excavated
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/claripy-8.18.10.25-py3.5.egg/claripy/ast/base.py", line 831, in _excavate_ite
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/claripy-8.18.10.25-py3.5.egg/claripy/ast/base.py", line 831, in <listcomp>
RecursionError: maximum recursion depth exceeded

test.zip The testing binary is attached.

rhelmot commented 5 years ago

The test case isn’t much good without the binary to run it on...

However, are you running on the latest master branch? #106 should have solved this class of issues for good.

winter2020 commented 5 years ago

The test case isn’t much good without the binary to run it on...

However, are you running on the latest master branch? #106 should have solved this class of issues for good.

My angr and claripy is up to date.

rhelmot commented 5 years ago

Then can I have your binary so I can run your testcase?

rhelmot commented 5 years ago

Ah, whoops, you just edited your first post. I’ll get to this in the morning.

winter2020 commented 5 years ago

Ah, whoops, you just edited your first post. I’ll get to this in the morning.

Yes, sure. Thanks.

ltfish commented 5 years ago

@benben2301 It seems that your claripy is the PyPI version (because of /home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/claripy-8.18.10.25-py3.5.egg/ in your error log). Our latest GitHub version contains a fix for this issue you reported. Please try the latest master branch of claripy on GitHub and see if this issue is resolved. Thanks.

rhelmot commented 5 years ago

hm, nope, it looks like this function is in fact still recursive after that PR.

@danse-macabre can you please convert excavate_ite to be non-recursive? If you're busy, let us know and one of us will pick it up when we have time.

danse-macabre commented 5 years ago

No problem, I'll give it a shot in a day or two.

winter2020 commented 5 years ago

Yes, the latest version still has the problem.

winter2020 commented 5 years ago

The new error is raised.

Traceback (most recent call last):
  File "checker.py", line 249, in <module>
    check(test_program)
  File "checker.py", line 149, in check
    vfg = proj.analyses.VFG(cfg, function_start=proj.entry, context_sensitivity_level=2, interfunction_level=4, remove_options={angr.options.OPTIMIZE_IR})
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/angr-8.18.10.25-py3.5.egg/angr/analyses/analysis.py", line 108, in __call__
    oself.__init__(*args, **kwargs)
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/angr-8.18.10.25-py3.5.egg/angr/analyses/vfg.py", line 359, in __init__
    self._analyze()
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/angr-8.18.10.25-py3.5.egg/angr/analyses/forward_analysis.py", line 552, in _analyze
    self._analysis_core_baremetal()
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/angr-8.18.10.25-py3.5.egg/angr/analyses/forward_analysis.py", line 662, in _analysis_core_baremetal
    self._pre_job_handling(job_info.job)
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/angr-8.18.10.25-py3.5.egg/angr/analyses/vfg.py", line 654, in _pre_job_handling
    job.sim_successors, _, restart_analysis = self._get_simsuccessors(input_state, addr)
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/angr-8.18.10.25-py3.5.egg/angr/analyses/vfg.py", line 1338, in _get_simsuccessors
    sim_successors = self.project.factory.successors(state, jumpkind=jumpkind, num_inst=num_inst)
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/angr-8.18.10.25-py3.5.egg/angr/factory.py", line 49, in successors
    return self.project.engines.successors(*args, **kwargs)
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/angr-8.18.10.25-py3.5.egg/angr/engines/hub.py", line 128, in successors
    r = engine.process(state, **kwargs)
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/angr-8.18.10.25-py3.5.egg/angr/engines/vex/engine.py", line 135, in process
    opt_level=opt_level)
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/angr-8.18.10.25-py3.5.egg/angr/engines/engine.py", line 55, in process
    self._process(new_state, successors, *args, **kwargs)
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/angr-8.18.10.25-py3.5.egg/angr/engines/vex/engine.py", line 185, in _process
    self._handle_irsb(state, successors, irsb, skip_stmts, last_stmt, whitelist)
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/angr-8.18.10.25-py3.5.egg/angr/engines/vex/engine.py", line 335, in _handle_irsb
    ret_state = exit_state.copy()
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/angr-8.18.10.25-py3.5.egg/angr/sim_state.py", line 453, in copy
    c_plugins = self._copy_plugins()
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/angr-8.18.10.25-py3.5.egg/angr/sim_state.py", line 440, in _copy_plugins
    out[n] = p.copy(memo)
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/angr-8.18.10.25-py3.5.egg/angr/state_plugins/plugin.py", line 58, in inner
    c = f(self, memo, **kwargs)
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/angr-8.18.10.25-py3.5.egg/angr/state_plugins/symbolic_memory.py", line 71, in copy
    mem=self.mem.branch(),
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/angr-8.18.10.25-py3.5.egg/angr/storage/paged_memory.py", line 369, in branch
    new_name_mapping = self._name_mapping.branch() if options.REVERSE_MEMORY_NAME_MAP in self.state.options else self._name_mapping
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/cooldict-1.3-py3.5.egg/cooldict.py", line 462, in branch
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/cooldict-1.3-py3.5.egg/cooldict.py", line 400, in __init__
  File "/home/wgh/.virtualenvs/angr/lib/python3.5/site-packages/cooldict-1.3-py3.5.egg/cooldict.py", line 205, in flatten
AttributeError: 'dict' object has no attribute 'iterkeys'
zardus commented 5 years ago

Hmm... This seems like it might be in "my" code. Let me check real quick.

zardus commented 5 years ago

Works fine for me:

(angr) yans@areth ~/code/angr/tmp $ ipython --pdb check.py 
Automatic calling is: Smart
WARNING | 2018-11-06 21:27:52,304 | angr.analyses.cfg.cfg_emulated | `advanced backward slicing` and `symbolic back traversal` are deprecated.
WARNING | 2018-11-06 21:27:52,304 | angr.analyses.cfg.cfg_emulated | Please use `resolve_indirect_jumps` to resolve indirect jumps using different resolvers instead.
WARNING | 2018-11-06 21:27:52,341 | angr.state_plugins.symbolic_memory | Memory address 0x7ffffffffff0000 has an unspecified value; Generating an unconstrained value of 8 bytes.
CRITICAL | 2018-11-06 21:27:52,380 | angr.sim_state | The name state.se is deprecated; please use state.solver.
WARNING | 2018-11-06 21:27:53,920 | angr.state_plugins.symbolic_memory | Memory address 0x700000000 has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:27:54,085 | angr.state_plugins.symbolic_memory | Memory address 0x700000064 has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:27:54,958 | angr.analyses.vfg | The given CFG is not normalized, which might impact the performance/accuracy of the VFG analysis.
WARNING | 2018-11-06 21:27:55,026 | angr.state_plugins.symbolic_memory | Register rdx has an unspecified value; Generating an unconstrained value of 8 bytes.
WARNING | 2018-11-06 21:27:55,035 | angr.state_plugins.symbolic_memory | Register rax has an unspecified value; Generating an unconstrained value of 8 bytes.
WARNING | 2018-11-06 21:27:57,008 | angr.state_plugins.symbolic_memory | Register r15 has an unspecified value; Generating an unconstrained value of 8 bytes.
WARNING | 2018-11-06 21:27:57,014 | angr.state_plugins.symbolic_memory | Register r14 has an unspecified value; Generating an unconstrained value of 8 bytes.
WARNING | 2018-11-06 21:27:57,027 | angr.state_plugins.symbolic_memory | Register r13 has an unspecified value; Generating an unconstrained value of 8 bytes.
WARNING | 2018-11-06 21:27:57,033 | angr.state_plugins.symbolic_memory | Register r12 has an unspecified value; Generating an unconstrained value of 8 bytes.
WARNING | 2018-11-06 21:27:57,043 | angr.state_plugins.symbolic_memory | Register rbx has an unspecified value; Generating an unconstrained value of 8 bytes.
WARNING | 2018-11-06 21:27:58,099 | angr.state_plugins.symbolic_memory | Memory address 0x700000064 has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:27:59,478 | angr.state_plugins.symbolic_memory | Memory address 0x700000065 has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:00,642 | angr.state_plugins.symbolic_memory | Memory address 0x700000066 has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:01,707 | angr.state_plugins.symbolic_memory | Memory address 0x700000067 has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:02,812 | angr.state_plugins.symbolic_memory | Memory address 0x700000068 has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:03,825 | angr.state_plugins.symbolic_memory | Memory address 0x700000069 has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:04,818 | angr.state_plugins.symbolic_memory | Memory address 0x70000006a has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:05,852 | angr.state_plugins.symbolic_memory | Memory address 0x70000006b has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:06,895 | angr.state_plugins.symbolic_memory | Memory address 0x70000006c has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:07,980 | angr.state_plugins.symbolic_memory | Memory address 0x70000006d has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:09,222 | angr.state_plugins.symbolic_memory | Memory address 0x70000006e has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:10,248 | angr.state_plugins.symbolic_memory | Memory address 0x70000006f has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:11,270 | angr.state_plugins.symbolic_memory | Memory address 0x700000070 has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:12,317 | angr.state_plugins.symbolic_memory | Memory address 0x700000071 has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:13,371 | angr.state_plugins.symbolic_memory | Memory address 0x700000072 has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:14,447 | angr.state_plugins.symbolic_memory | Memory address 0x700000073 has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:15,540 | angr.state_plugins.symbolic_memory | Memory address 0x700000074 has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:16,670 | angr.state_plugins.symbolic_memory | Memory address 0x700000075 has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:17,831 | angr.state_plugins.symbolic_memory | Memory address 0x700000076 has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:19,183 | angr.state_plugins.symbolic_memory | Memory address 0x700000077 has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:20,325 | angr.state_plugins.symbolic_memory | Memory address 0x700000078 has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:21,515 | angr.state_plugins.symbolic_memory | Memory address 0x700000079 has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:22,727 | angr.state_plugins.symbolic_memory | Memory address 0x70000007a has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:23,980 | angr.state_plugins.symbolic_memory | Memory address 0x70000007b has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:25,227 | angr.state_plugins.symbolic_memory | Memory address 0x70000007c has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:26,502 | angr.state_plugins.symbolic_memory | Memory address 0x70000007d has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:27,799 | angr.state_plugins.symbolic_memory | Memory address 0x70000007e has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:29,138 | angr.state_plugins.symbolic_memory | Memory address 0x70000007f has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:30,757 | angr.state_plugins.symbolic_memory | Memory address 0x700000080 has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:32,156 | angr.state_plugins.symbolic_memory | Memory address 0x700000081 has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:33,549 | angr.state_plugins.symbolic_memory | Memory address 0x700000082 has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:34,976 | angr.state_plugins.symbolic_memory | Memory address 0x700000083 has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:36,422 | angr.state_plugins.symbolic_memory | Memory address 0x700000084 has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:37,959 | angr.state_plugins.symbolic_memory | Memory address 0x700000085 has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:39,486 | angr.state_plugins.symbolic_memory | Memory address 0x700000086 has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:41,309 | angr.state_plugins.symbolic_memory | Memory address 0x700000087 has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:42,891 | angr.state_plugins.symbolic_memory | Memory address 0x700000088 has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:44,511 | angr.state_plugins.symbolic_memory | Memory address 0x700000089 has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:46,190 | angr.state_plugins.symbolic_memory | Memory address 0x70000008a has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:47,924 | angr.state_plugins.symbolic_memory | Memory address 0x70000008b has an unspecified value; Generating an unconstrained value of 1 bytes.
WARNING | 2018-11-06 21:28:49,681 | angr.state_plugins.symbolic_memory | Memory address 0x70000008c has an unspecified value; Generating an unconstrained value of 1 bytes.

Are you sure you have all the latest of everything?

winter2020 commented 5 years ago

Yes, everything is up-to-date, still the same error. Could you check whether we are using the same binary? the binary I am using is attached. test.zip

rhelmot commented 5 years ago

This was fixed 20 days ago but hasn't been uploaded to pypi. Install cooldict from https://github.com/zardus/cooldict. I'll do a new release as soon as I have time.

winter2020 commented 5 years ago

Yes, I have updated cooldict, vsa is working now.