search

angr / rex

Shellphish's automated exploitation engine, originally created for the Cyber Grand Challenge.
BSD 2-Clause "Simplified" License
635 stars 102 forks source link

No module named 'archr.analyzers' #80

Closed yh570 closed 3 years ago

yh570 commented 3 years ago

Hi, I met a problem when I import rex in python. I successfully install rex with pip install ., but I got an error message whish is shown below:

>>> import rex
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/yh570/driller/vir/lib/python3.7/site-packages/rex/__init__.py", line 2, in <module>
    from rex.crash import Crash, NonCrashingInput
  File "/home/yh570/driller/vir/lib/python3.7/site-packages/rex/crash.py", line 18, in <module>
    from archr.analyzers.angr_state import SimArchrProcMount
ModuleNotFoundError: No module named 'archr.analyzers'

I checked archr which is correctly installed, but it looks like it's not the correct package which rex requires, I'm confused about this situation, please give me some advice about this problem. Thank you.

twizmwazin commented 3 years ago

To install angr components from source, see the angr/angr-dev repository. You need to install all of the angr components from source, not pypi.

yh570 commented 3 years ago

Thank you for your comments, I reinstalled angr-dev followed by the setup instruction from angr-dev repo as ./setup.sh -i -e angr, and I checked it works well with angr-docs examples. I then re-install rex and its components, but I still meet a problem with `import rex', which is shown below:

>>> import rex
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/yh570/.virtualenvs/angr/local/lib/python3.7/site-packages/rex/__init__.py", line 2, in <module>
    from rex.crash import Crash, NonCrashingInput
  File "/home/yh570/.virtualenvs/angr/local/lib/python3.7/site-packages/rex/crash.py", line 20, in <module>
    from .exploit import CannotExploit, CannotExplore, ExploitFactory, CGCExploitFactory
  File "/home/yh570/.virtualenvs/angr/local/lib/python3.7/site-packages/rex/exploit/__init__.py", line 4, in <module>
    from .exploit import Exploit, ExploitException
  File "/home/yh570/.virtualenvs/angr/local/lib/python3.7/site-packages/rex/exploit/exploit.py", line 12, in <module>
    from ..scripter import Scripter
  File "/home/yh570/.virtualenvs/angr/local/lib/python3.7/site-packages/rex/scripter/__init__.py", line 10, in <module>
    loader=jinja2.PackageLoader('rex', 'scripter/templates'),
  File "/home/yh570/.virtualenvs/angr/local/lib/python3.7/site-packages/jinja2/loaders.py", line 310, in __init__
    f"The {package_name!r} package was not installed in a"
ValueError: The 'rex' package was not installed in a way that PackageLoader understands.

I installed rex and its components by downloading their git repo and then pip install .

twizmwazin commented 3 years ago

Did rex install correctly or did the pip install error out with dependency issues? rex has multiple dependencies that need to be fetched manually or with setup.sh. pip freeze will show what is installed in your virtualenv.

yh570 commented 3 years ago

Did rex install correctly or did the pip install error out with dependency issues? rex has multiple dependencies that need to be fetched manually or with setup.sh. pip freeze will show what is installed in your virtualenv.

I didn't get any error messages during the installation. The log of pip freeze is shown below:

-e git+https://github.com/angr/ailment@bef6268dd3d4ea9c251fd24f8a301375771d9dd7#egg=ailment
ana==0.5
-e git+https://github.com/angr/angr@bfba2af1ea2eb941001339f47a1264a685c60eec#egg=angr
-e git+https://github.com/angr/angr-management@7033aa25957d8d59cea7ba10e296d38b4b6678b7#egg=angr_management
angr-pwntools==4.5.0
-e git+https://github.com/angr/angr-targets@6ebf346acf4273b8516fa1c802932b9e09d4448b#egg=angr_targets
-e git+https://github.com/angr/angrop@fa030fd4534d7abbaf2321c7877812ecb8232488#egg=angrop
-e git+https://github.com/angr/archinfo@b150db4c0a939140966df8b0056b6deb5b07efbf#egg=archinfo
-e git+https://github.com/angr/archr@ba31e7b5f4b4df515c988155dd2320ee118036c9#egg=archr
astroid==2.6.6
avatar2==1.2.2
backcall==0.1.0
bcrypt==3.2.0
bitstring==3.1.5
cachetools==3.1.0
capstone==4.0.1
certifi==2019.6.16
cffi==1.14.6
chardet==3.0.4
-e git+https://github.com/angr/claripy@36c640346a822a1950ca43d6d75678e33c731832#egg=claripy
-e git+https://github.com/angr/cle@3909a5ffdb1d4126e0ef359e8013e79350b12a92#egg=cle
colored-traceback==0.3.0
compilerex @ file:///home/yh570/driller/rex/compilerex
configparser==3.8.1
coverage==5.5
CppHeaderParser==2.7.4
cryptography==3.4.7
debugpy==1.4.1
decorator==4.4.0
docker==4.0.2
dpkt==1.9.2
enum34==1.1.6
flaky==3.7.0
future==0.17.1
getmac==0.8.2
gitdb2==2.0.5
GitPython==2.1.11
greenlet==1.1.1
idalink==0.12
idna==2.8
importlib-metadata==3.10.1
intervaltree==3.0.2
ipdb==0.13.9
ipykernel==6.0.3
ipython==7.26.0
ipython-genutils==0.2.0
isort==4.3.21
itanium-demangler==1.0
jedi==0.18.0
Jinja2==3.0.1
jupyter-client==6.1.12
jupyter-core==4.7.1
keystone-engine==0.9.2
lazy-object-proxy==1.4.1
Mako==1.1.4
MarkupSafe==2.0.1
matplotlib-inline==0.1.2
mccabe==0.6.1
-e git+https://git:@github.com/rhelmot/monkeyhex@2718ae888d05c0827af3aca9bb46d25f773edfc2#egg=monkeyhex
-e git+https://git:@github.com/zardus/mulpyplexer@2f3c8761650b09a1ff8a14ef64c346ec0b610b42#egg=mulpyplexer
nampa==0.1.1
nclib==1.0.0rc4
networkx==2.2
ninja==1.10.2
nose==1.3.7
nose-timer==1.0.1
npyscreen==4.10.5
packaging==21.0
paramiko==2.7.2
parse==1.12.1
parso==0.8.2
patchelf-wrapper==1.2.0
pefile==2018.8.8
pexpect==4.7.0
pickleshare==0.7.5
plumbum==1.6.7
ply==3.11
posix-ipc==1.0.4
povsim @ file:///home/yh570/driller/rex/povsim
progressbar==2.5
progressbar2==3.53.1
prompt-toolkit==2.0.9
protobuf==3.17.3
psutil==5.6.3
ptyprocess==0.6.0
pycparser==2.19
pyelftools @ git+https://github.com/eliben/pyelftools@ab444d982d1849191e910299a985989857466620
pygdbmi==0.9.0.2
Pygments==2.4.2
PyLink==0.3.3
pylint==2.9.6
PyNaCl==1.4.0
pyparsing==2.4.7
pyqodeng.core==0.0.3
pyserial==3.4
PySide2==5.15.2
PySMT==0.9.1.dev139
PySocks==1.7.1
-e git+https://github.com/angr/pysoot@d08dc569ec35796ccea5509b3e04b74967bcfd48#egg=pysoot
python-dateutil==2.8.2
python-magic==0.4.24
python-utils==2.5.6
-e git+https://github.com/angr/pyvex@4a37c8330435f7323036e2bc08f4d6271ed24eae#egg=pyvex
pyxdg==0.27
pyzmq==22.2.1
qtconsole==5.1.1
QtPy==1.9.0
qtterm==0.5.1
requests==2.22.0
rex @ file:///home/yh570/driller/rex/rex
ROPGadget==6.6
rpyc==4.0.2
shellphish-qemu==0.12.3
shiboken2==5.15.2
six==1.12.0
smmap2==2.0.5
sortedcontainers==2.1.0
SQLAlchemy==1.4.22
toml==0.10.2
tornado==6.1
tqdm==4.62.0
tracer @ file:///home/yh570/driller/rex/tracer
traitlets==4.3.2
typed-ast==1.4.0
typing-extensions==3.10.0.0
unicorn==1.0.2rc4
urllib3==1.25.3
wcwidth==0.1.7
websocket-client==0.56.0
wrapt==1.11.2
z3-solver==4.8.5.0
zipp==3.5.0

Is there any setup instruction for rex so I can check if I incorrectly install some components?

twizmwazin commented 3 years ago

Instead of cloning/downloading repositories and then installing them (resulting in lines like rex @ file:///home/yh570/driller/rex/rex), please install them using setup.sh in angr-dev. For example, ./setup.sh -e angr povsim to install povsim. It will clone and install the repo as it does with other angr repos, and any packaging idiosyncrasies should be mitigated since this would match the typical development setup.

yh570 commented 3 years ago

I appreciate your help so much. I successfully installed rex by using commands ./setup.sh -i -e angr capstone unicorn archinfo vex pyvex cle claripy simuvex angr angr-management angr-doc angrop tracer compilerex povsim rex patcherex

However, I still met a problem with installing shellphish-qemu. By using the command ./setup.sh -i -e angr shellphish-qemu, the installation was failed with an error message as below:

    ERROR: Command errored out with exit status 1:
     command: /home/yh570/.virtualenvs/angr/bin/python3 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/yh570/driller/rex/angr-dev/shellphish-qemu/setup.py'"'"'; __file__='"'"'/home/yh570/driller/rex/angr-dev/shellphish-qemu/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
         cwd: /home/yh570/driller/rex/angr-dev/shellphish-qemu/
    Complete output (42 lines):
    running develop
    Cloning CGC QEMU
    Cloning Linux QEMU
    Building Tracer QEMU
    rm -f *.timestamp
    make -C tests/tcg clean
    make[1]: Entering directory '/home/yh570/driller/rex/angr-dev/shellphish-qemu/shellphish-qemu-cgc-base/tests/tcg'
    rm -f *.timestamp
    rm -f *~ *.o test-i386.out test-i386.ref \
               test-x86_64.log test-x86_64.ref qruncom test_path hello-i386 linux-test testthread sha1-i386 test-i386 test-i386-fprem test-mmap  run-test-x86_64
make[1]: Leaving directory '/home/yh570/driller/rex/angr-dev/shellphish-qemu/shellphish-qemu-cgc-base/tests/tcg'
    rm -rf tests/check-qdict tests/check-qfloat tests/check-qint tests/check-qstring tests/check-qlist tests/check-qjson tests/test-qmp-output-visitor tests/test-qmp-input-visitor tests/test-qmp-input-strict tests/test-qmp-commands tests/test-string-input-visitor tests/test-string-output-visitor tests/test-qmp-event tests/test-opts-visitor tests/test-coroutine tests/test-visitor-serialization tests/test-iov tests/test-aio tests/test-rfifolock tests/test-throttle tests/test-thread-pool tests/test-hbitmap tests/test-x86-cpuid tests/test-cutils tests/test-mul64 tests/test-int128 tests/rcutorture tests/test-rcu-list tests/test-bitops tests/check-qom-interface tests/test-qemu-opts tests/test-write-threshold tests/*.o tests/qemu-iotests/socket_scm_helper
    rm -rf tests/ac97-test tests/ahci-test tests/bios-tables-test tests/boot-order-test tests/display-vga-test tests/drive_del-test tests/e1000-test tests/eepro100-test tests/endianness-test tests/es1370-test tests/fdc-test tests/fw_cfg-test tests/hd-geo-test tests/i440fx-test tests/i82801b11-test tests/ide-test tests/intel-hda-test tests/ioh3420-test tests/ipoctal232-test tests/ne2000-test tests/nvme-test tests/pc-cpu-test tests/pcnet-test tests/pvpanic-test tests/qom-test tests/rtc-test tests/rtl8139-test tests/spapr-phb-test tests/tmp105-test tests/tpci200-test tests/usb-hcd-ehci-test tests/usb-hcd-ohci-test tests/usb-hcd-uhci-test tests/usb-hcd-xhci-test tests/vhost-user-test tests/virtio-balloon-test tests/virtio-blk-test tests/virtio-console-test tests/virtio-net-test tests/virtio-rng-test tests/virtio-scsi-test tests/virtio-serial-test tests/vmxnet3-test tests/wdt_ib700-test
    rm -f config.mak op-i386.h opc-i386.h gen-op-i386.h op-arm.h opc-arm.h gen-op-arm.h
    rm -f qemu-options.def
    find . \( -name '*.l[oa]' -o -name '*.so' -o -name '*.dll' -o -name '*.mo' -o -name '*.[oda]' \) -type f -exec rm {} +
    rm -f  qemu-bridge-helper qemu-ga TAGS cscope.* *.pod *~ */*~
    rm -f fsdev/*.pod
    rm -rf .libs */.libs
    rm -f qemu-img-cmds.h
    rm -f trace/generated-tracers-dtrace.dtrace*
    rm -f trace/generated-tracers-dtrace.h*
    rm -f config-host.h config-host.h-timestamp qemu-options.def qemu-options.def-timestamp qmp-commands.h qmp-commands.h-timestamp qapi-types.h qapi-types.h-timestamp qapi-visit.h qapi-visit.h-timestamp qapi-event.h qapi-event.h-timestamp trace/generated-events.h trace/generated-events.h-timestamp trace/generated-tracers.h trace/generated-tracers.h-timestamp trace/generated-tcg-tracers.h trace/generated-tcg-tracers.h-timestamp trace/generated-helpers-wrappers.h trace/generated-helpers-wrappers.h-timestamp trace/generated-helpers.h trace/generated-helpers.h-timestamp tests/test-qapi-types.h tests/test-qapi-types.h-timestamp tests/test-qapi-visit.h tests/test-qapi-visit.h-timestamp tests/test-qmp-commands.h tests/test-qmp-commands.h-timestamp tests/test-qapi-event.h tests/test-qapi-event.h-timestamp
    rm -f qmp-marshal.c qmp-marshal.c-timestamp qapi-types.c qapi-types.c-timestamp qapi-visit.c qapi-visit.c-timestamp qapi-event.c qapi-event.c-timestamp trace/generated-events.c trace/generated-events.c-timestamp trace/generated-tracers.c trace/generated-tracers.c-timestamp trace/generated-helpers.c trace/generated-helpers.c-timestamp
    rm -rf qapi-generated
    rm -rf qga/qapi-generated
    for d in i386-linux-user ; do \
    if test -d $d; then make -C $d clean || exit 1; fi; \
    rm -f $d/qemu-options.def; \
            done
    make[1]: Entering directory '/home/yh570/driller/rex/angr-dev/shellphish-qemu/shellphish-qemu-cgc-base/i386-linux-user'
    rm -f *.timestamp
    rm -f *.a *~ qemu-i386
    rm -f
    rm -f hmp-commands.h qmp-commands-old.h gdbstub-xml.c
    make[1]: Leaving directory '/home/yh570/driller/rex/angr-dev/shellphish-qemu/shellphish-qemu-cgc-base/i386-linux-user'

    ERROR: invalid trace backends
           Please choose supported trace backends.

    Configuring CGC tracer qemu...
    error: Unable to configure shellphish-qemu-cgc-tracer

I also tried running python setup.py install in angr virtual environment but it's failed. I tried pip install git+https://github.com/shellphish/shellphish-qemu which is successful, but when I run the example test_rex.py in tests folder, I got the error message:

archr.analyzers.qemu_tracer.QEMUTracerError: the target didn't crash inside qemu! Make sure you launch it correctly!
command: /tmp/archr_local__j6tnc66/shellphish_qemu/fire /tmp/archr_local__j6tnc66/shellphish_qemu/shellphish-qemu-cgc-tracer -C /tmp/tracer_target_bzm9j3dr -d exec -D /tmp/tracer-rqyzhqn5.trace -magicdump /tmp/tracer-rqyzhqn5.magic -m 8G -- /home/yh570/driller/rex/rex/tests/../../binaries/tests/i386/arbitrary_transmit

I checked the file /tmp/archr_local__j6tnc66/shellphish_qemu/fire but it does not exist, so I think the shellphish-qemu is incorrectly installed.

twizmwazin commented 3 years ago

shellphish-qemu requires build deps for qemu if you wish to install it from source, however that package can be installed from PyPI. Capstone and unicorn do not need to be installed separately, they too can be pulled from PyPI. Simuvex was merged years ago into angr and does not need to be installed.

yh570 commented 3 years ago

I uninstalled shellphish_qemu and then re-installed it with pip install shellphish_qemu, which works fine. But I still get the same error message when ran the test_rex.py, the error message is shown below:

(angr) yh570:~/driller/rex/rex/tests$ python test_rex.py                                                                         
test_arbitrary_transmit
DEBUG   | 2021-08-09 05:52:43,506 | archr.targets | Running command: 'mkdir' '/tmp/tracer_target_1p0ftv9c'
DEBUG   | 2021-08-09 05:52:43,535 | archr.analyzers.qemu_tracer | launch QEMU with command: /tmp/archr_local_gvqdzmgl/shellphish_qemu/fire /tmp/archr_local_gvqdzmgl/shellphish_qemu/shellphish-qemu-cgc-tracer -C /tmp/tracer_target_1p0ftv9c -d exec -D /tmp/tracer-stf0448z.trace -magicdump /tmp/tracer-stf0448z.magic -m 8G -- /home/yh570/driller/rex/rex/tests/../../binaries/tests/i386/arbitrary_transmit
DEBUG   | 2021-08-09 05:52:43,536 | archr.targets | Running command: '/tmp/archr_local_gvqdzmgl/shellphish_qemu/fire' '/tmp/archr_local_gvqdzmgl/shellphish_qemu/shellphish-qemu-cgc-tracer' '-C' '/tmp/tracer_target_1p0ftv9c' '-d' 'exec' '-D' '/tmp/tracer-stf0448z.trace' '-magicdump' '/tmp/tracer-stf0448z.magic' '-m' '8G' '--' '/home/yh570/driller/rex/rex/tests/../../binaries/tests/i386/arbitrary_transmit'
DEBUG   | 2021-08-09 05:52:43,547 | archr.target.actions | [OpenChannelAction] openning channel: stdio
DEBUG   | 2021-08-09 05:52:43,547 | archr.target.actions | [SendAction] sending data to channel stdio: b'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA'
INFO    | 2021-08-09 05:52:43,547 | archr.log | ======= Sending 36 bytes =======

INFO    | 2021-08-09 05:52:43,548 | archr.log | >> AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA%

DEBUG   | 2021-08-09 05:52:43,583 | archr.targets | Running command: 'rm' '-rf' '/tmp/tracer_target_1p0ftv9c'
Traceback (most recent call last):
  File "test_rex.py", line 456, in <module>
    run_all()
  File "test_rex.py", line 442, in run_all
    all_functions[f]()
  File "test_rex.py", line 344, in test_arbitrary_transmit
    _do_arbitrary_transmit_test_for("tests/i386/arbitrary_transmit")
  File "test_rex.py", line 328, in _do_arbitrary_transmit_test_for
    crash = rex.Crash(target, inp, fast_mode=True, rop_cache_path=os.path.join(cache_location, os.path.basename(binary)))
  File "/home/yh570/driller/rex/angr-dev/rex/rex/crash.py", line 612, in __init__
    self._initialize()
  File "/home/yh570/driller/rex/angr-dev/rex/rex/crash.py", line 994, in _initialize
    self.concrete_trace()
  File "/home/yh570/driller/rex/angr-dev/rex/rex/crash.py", line 406, in concrete_trace
    taint=taint)
  File "/home/yh570/driller/rex/angr-dev/rex/rex/crash_tracer/full_tracer.py", line 17, in concrete_trace
    pre_fire_hook=pre_fire_hook, delay=delay, actions=actions, taint=taint)
  File "/home/yh570/driller/rex/angr-dev/archr/archr/analyzers/__init__.py", line 71, in fire
    self._fire_testcase(flight, channel=channel)
  File "/usr/lib/python3.7/contextlib.py", line 119, in __exit__
    next(self.gen)
  File "/home/yh570/driller/rex/angr-dev/archr/archr/analyzers/qemu_tracer.py", line 143, in fire_context
    "command: %s" % ' '.join(target_cmd))
archr.analyzers.qemu_tracer.QEMUTracerError: the target didn't crash inside qemu! Make sure you launch it correctly!
command: /tmp/archr_local_gvqdzmgl/shellphish_qemu/fire /tmp/archr_local_gvqdzmgl/shellphish_qemu/shellphish-qemu-cgc-tracer -C /tmp/tracer_target_1p0ftv9c -d exec -D /tmp/tracer-stf0448z.trace -magicdump /tmp/tracer-stf0448z.magic -m 8G -- /home/yh570/driller/rex/rex/tests/../../binaries/tests/i386/arbitrary_transmit
yh570 commented 3 years ago

I solved the problem, now I'm working on understanding how rex works. Thanks for your help.

aesophor commented 2 years ago

@yh570 Hey sorry for reviving this thread, but I've encountered this very same problem. I tried reinstalling rex, archr and shellphish_qemu but it still gives me the same error you had. Do you remember how you solved it? Thanks.

/home/aesophor/Code/rex [git::master] [aesophor@aesophor-vm] [13:53]
> python tests/test_rex.py
test_arbitrary_transmit
DEBUG   | 2022-04-29 13:53:39,815 | archr.targets | Running command: 'mkdir' '/tmp/tracer_target_4nckdczj'
DEBUG   | 2022-04-29 13:53:39,819 | archr.targets | Running command: 'chmod' '777' '/tmp/tracer_target_4nckdczj'
DEBUG   | 2022-04-29 13:53:39,823 | archr.analyzers.qemu_tracer | launch QEMU with command: /tmp/archr_local_t3man2vm/shellphish_qemu/fire /tmp/archr_local_t3man2vm/shellphish_qemu/shellphish-qemu-cgc-tracer -C /tmp/tracer_target_4nckdczj -d exec -D /tmp/tracer-ouciu7a1.trace -magicdump /tmp/tracer-ouciu7a1.magic -m 8G -- /home/aesophor/Code/rex/tests/../../binaries/tests/i386/arbitrary_transmit
DEBUG   | 2022-04-29 13:53:39,823 | archr.targets | Running command: '/tmp/archr_local_t3man2vm/shellphish_qemu/fire' '/tmp/archr_local_t3man2vm/shellphish_qemu/shellphish-qemu-cgc-tracer' '-C' '/tmp/tracer_target_4nckdczj' '-d' 'exec' '-D' '/tmp/tracer-ouciu7a1.trace' '-magicdump' '/tmp/tracer-ouciu7a1.magic' '-m' '8G' '--' '/home/aesophor/Code/rex/tests/../../binaries/tests/i386/arbitrary_transmit'
DEBUG   | 2022-04-29 13:53:39,826 | archr.target.actions | [OpenChannelAction] openning channel: stdio
DEBUG   | 2022-04-29 13:53:39,826 | archr.target.actions | [SendAction] sending data to channel stdio: b'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA'
INFO    | 2022-04-29 13:53:39,826 | archr.log | ======= Sending 36 bytes =======

INFO    | 2022-04-29 13:53:39,826 | archr.log | >> AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA%

DEBUG   | 2022-04-29 13:53:39,828 | archr.analyzers.qemu_tracer | Qemu tracer returned with code=1 timed_out=False crashed=None signal=None
DEBUG   | 2022-04-29 13:53:39,829 | archr.targets | Running command: 'rm' '-rf' '/tmp/tracer_target_4nckdczj'
Traceback (most recent call last):
  File "tests/test_rex.py", line 464, in <module>
    run_all()
  File "tests/test_rex.py", line 450, in run_all
    all_functions[f]()
  File "tests/test_rex.py", line 349, in test_arbitrary_transmit
    _do_arbitrary_transmit_test_for("tests/i386/arbitrary_transmit")
  File "tests/test_rex.py", line 332, in _do_arbitrary_transmit_test_for
    crash = rex.Crash(target, inp, fast_mode=True,
  File "/home/aesophor/.local/lib/python3.8/site-packages/rex/crash.py", line 612, in __init__
    self._initialize()
  File "/home/aesophor/.local/lib/python3.8/site-packages/rex/crash.py", line 994, in _initialize
    self.concrete_trace()
  File "/home/aesophor/.local/lib/python3.8/site-packages/rex/crash.py", line 402, in concrete_trace
    self.trace_result, self.core_registers = self.tracer.concrete_trace(testcase, channel,
  File "/home/aesophor/.local/lib/python3.8/site-packages/rex/crash_tracer/full_tracer.py", line 16, in concrete_trace
    r = self.tracer_bow.fire(testcase=testcase, channel=channel, save_core=True, record_magic=self._is_cgc,
  File "/home/aesophor/Code/archr/archr/analyzers/__init__.py", line 71, in fire
    self._fire_testcase(flight, channel=channel)
  File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__
    next(self.gen)
  File "/home/aesophor/Code/archr/archr/analyzers/qemu_tracer.py", line 156, in fire_context
    raise QEMUTracerError("the target didn't crash inside qemu or no corefile was created!" +
archr.analyzers.qemu_tracer.QEMUTracerError: the target didn't crash inside qemu or no corefile was created!Make sure you launch it correctly!
command: /tmp/archr_local_t3man2vm/shellphish_qemu/fire /tmp/archr_local_t3man2vm/shellphish_qemu/shellphish-qemu-cgc-tracer -C /tmp/tracer_target_4nckdczj -d exec -D /tmp/tracer-ouciu7a1.trace -magicdump /tmp/tracer-ouciu7a1.magic -m 8G -- /home/aesophor/Code/rex/tests/../../binaries/tests/i386/arbitrary_transmit
rhelmot commented 2 years ago

What is the output when you run shellphish-qemu-cgc-tracer /home/aesophor/Code/rex/tests/../../binaries/tests/i386/arbitrary_transmit by itself?

aesophor commented 2 years ago

@rhelmot Thanks for your reply! It turns out that I forgot to clone angr/binaries.

I've managed to get rex to run, but compilerex now says "libcgc.h" not found while dumping an exploit as a compiled POV.

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/aesophor/.local/lib/python3.8/site-packages/rex/exploit/cgc/cgc_exploit.py", line 138, in dump_binary
    compiled_result = compilerex.compile_from_string(c_code)
  File "/home/aesophor/.local/lib/python3.8/site-packages/compilerex/compilerex.py", line 132, in compile_from_string
    raise CompileError(res[1])
compilerex.compilerex.CompileError: b"/tmp/c_file_2obmgpmg.c:2:10: fatal error: 'libcgc.h' file not found\n#include <libcgc.h>\n         ^\n1 error generated.\n"

As a result, I installed it from CyberGrandChallenge/libcgc, and installed boolector from boolector. Now the headers exist on my disk.

/usr [aesophor@aesophor-vm] [19:01]
> ll /usr/include/libcgc.h
-rwxr-xr-x 1 root root 3.0K May  2 17:54 /usr/include/libcgc.h

/usr [aesophor@aesophor-vm] [19:01]
> ll /usr/local/include/boolector.h
-rw-r--r-- 1 root root 79K May  2 18:36 /usr/local/include/boolector.h

Running compilerex's clang again but this time stdbool.h is not found.

> /home/aesophor/.local/lib/python3.8/site-packages/compilerex/scripts/../bin/clang /tmp/c_file_bycgdq7b.c In file included from /tmp/c_file_bycgdq7b.c:4:
/usr/local/include/boolector.h:13:10: fatal error: 'stdbool.h' file not found
#include <stdbool.h>
         ^
1 error generated.

So I try to find all stdbool.h on my disk

> find . | grep 'stdbool.h'
./include/c++/9/tr1/stdbool.h
./lib/gcc/x86_64-linux-gnu/9/include/stdbool.h
./lib/gcc/x86_64-linux-gnu/7/include/stdbool.h
./lib/gcc/x86_64-w64-mingw32/9.3-posix/include/c++/tr1/stdbool.h
./lib/gcc/x86_64-w64-mingw32/9.3-posix/include/stdbool.h
./lib/gcc/x86_64-w64-mingw32/9.3-win32/include/c++/tr1/stdbool.h
./lib/gcc/x86_64-w64-mingw32/9.3-win32/include/stdbool.h
./lib/gcc/i686-w64-mingw32/9.3-posix/include/c++/tr1/stdbool.h
./lib/gcc/i686-w64-mingw32/9.3-posix/include/stdbool.h
./lib/gcc/i686-w64-mingw32/9.3-win32/include/c++/tr1/stdbool.h
./lib/gcc/i686-w64-mingw32/9.3-win32/include/stdbool.h
./lib/llvm-10/lib/clang/10.0.0/include/stdbool.h

Running compilerex's clang again gives typedef redefinition errors.

/home/aesophor/.local/lib/python3.8/site-packages/compilerex/scripts/../bin/clang /tmp/c_file_bycgdq7b.c -o out -I/lib/llvm-10/lib/clang/10.0.0/include
In file included from /tmp/c_file_bycgdq7b.c:4:
In file included from /usr/local/include/boolector.h:15:
In file included from /usr/include/stdio.h:33:
/lib/llvm-10/lib/clang/10.0.0/include/stddef.h:46:23: error: typedef redefinition with different types
      ('unsigned int' vs 'unsigned long')
typedef __SIZE_TYPE__ size_t;
                      ^
/usr/include/libcgc.h:10:27: note: previous definition is here
typedef long unsigned int size_t;
                          ^
In file included from /tmp/c_file_bycgdq7b.c:4:
In file included from /usr/local/include/boolector.h:15:
/usr/include/stdio.h:77:19: error: typedef redefinition with different types ('__ssize_t' (aka 'int') vs
      'long')
typedef __ssize_t ssize_t;
                  ^
/usr/include/libcgc.h:11:25: note: previous definition is here
typedef long signed int ssize_t;
                        ^
/tmp/c_file_bycgdq7b.c:196:25: warning: implicitly declaring library function 'calloc' with type 'void
      *(unsigned int, unsigned int)'
  char *received_data = calloc(recv_buf_len, 1);
                        ^
/tmp/c_file_bycgdq7b.c:196:25: note: please include the header <stdlib.h> or explicitly provide a
      declaration for 'calloc'
/tmp/c_file_bycgdq7b.c:202:16: warning: passing 'char *' to parameter of type 'unsigned char *' converts
      between pointers to integer types with different sign [-Wpointer-sign]
  receive_n(0, received_data, recv_buf_len);
               ^~~~~~~~~~~~~
/tmp/c_file_bycgdq7b.c:55:42: note: passing argument to parameter 'dst' here
size_t receive_n( int fd, unsigned char *dst, size_t n_bytes )
                                         ^
2 warnings and 2 errors generated.

At this point, I think I'm probably doing it completely wrong... Sorry for asking this dumb question but I can't work this out on my end. Thanks very much for your time...

rhelmot commented 2 years ago

Before we get into debugging this issue, why are you trying to compile your exploits as cgc binaries? this is pretty much never useful unless you are actively trying to participate in a competition using the cgc format. You probably want to output your exploits as python scripts.