shellphish / shellphish-afl

A pip wrapper around AFL.
BSD 2-Clause "Simplified" License
82 stars 51 forks source link

memfd patch not applied during pip installation from git #20

Closed SpaceMoehre closed 3 years ago

SpaceMoehre commented 3 years ago

I'm trying to setup the shellphuzz but it appears as the pip installation does not apply the memfd patch necessary for afl qemu mode. My Dockerfile:

FROM ubuntu:18.04

RUN cp /etc/apt/sources.list /etc/apt/sources.list~
RUN sed -Ei 's/^# deb-src /deb-src /' /etc/apt/sources.list
RUN apt-get update && apt-get install build-essential gcc-multilib debootstrap debian-archive-keyring libtool-bin bison python3 virtualenv git python3-dev automake -y
RUN apt-get build-dep qemu -y

RUN virtualenv -ppython3 /opt/venv
RUN /opt/venv/bin/pip install git+https://github.com/shellphish/driller

RUN /opt/venv/bin/pip install git+https://github.com/shellphish/shellphish-afl
RUN /opt/venv/bin/pip install git+https://github.com/shellphish/fuzzer

It runs without problems until it tries to execute afl's build_qemu_support.sh script:

  util/memfd.c:40:12: error: static declaration of 'memfd_create' follows non-static declaration
   static int memfd_create(const char *name, unsigned int flags)
              ^~~~~~~~~~~~
  In file included from /usr/include/x86_64-linux-gnu/bits/mman-linux.h:115:0,
                   from /usr/include/x86_64-linux-gnu/bits/mman.h:45,
                   from /usr/include/x86_64-linux-gnu/sys/mman.h:41,
                   from /tmp/pip-req-build-rwi9pkjk/bin/afl-unix/qemu_mode/qemu-2.10.0/include/sysemu/os-posix.h:29,
                   from /tmp/pip-req-build-rwi9pkjk/bin/afl-unix/qemu_mode/qemu-2.10.0/include/qemu/osdep.h:104,
                   from util/memfd.c:28:
  /usr/include/x86_64-linux-gnu/bits/mman-shared.h:46:5: note: previous declaration of 'memfd_create' was here
   int memfd_create (const char *__name, unsigned int __flags) __THROW;
       ^~~~~~~~~~~~
  /tmp/pip-req-build-rwi9pkjk/bin/afl-unix/qemu_mode/qemu-2.10.0/rules.mak:66: recipe for target 'util/memfd.o' failed
  make: *** [util/memfd.o] Error 1
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-req-build-rwi9pkjk/setup.py", line 123, in <module>
      scripts=['fetchlibs.sh'],
    File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
      cmd_obj.run()
    File "/opt/venv/lib/python3.6/site-packages/wheel/bdist_wheel.py", line 299, in run
      self.run_command('build')
    File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
      cmd_obj.run()
    File "/tmp/pip-req-build-rwi9pkjk/setup.py", line 102, in run
      self.execute(_setup_other_arch, (), msg="Setting up AFL-other-arch")
    File "/usr/lib/python3.6/distutils/cmd.py", line 335, in execute
      util.execute(func, args, msg, dry_run=self.dry_run)
    File "/usr/lib/python3.6/distutils/util.py", line 301, in execute
      func(*args)
    File "/tmp/pip-req-build-rwi9pkjk/setup.py", line 42, in _setup_other_arch
      if subprocess.check_call(['./build.sh'] + SUPPORTED_ARCHES, cwd=AFL_UNIX_INSTALL_PATH) != 0:
    File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['./build.sh', 'aarch64', 'x86_64', 'i386', 'arm', 'ppc', 'ppc64', 'mips', 'mipsel', 'mips64']' returned non-zero exit status 1.
  ----------------------------------------
  ERROR: Failed building wheel for shellphish-afl

The error code originates from the missing memfd patch. Even though the memfd.diff is included in this repo, it looks like its not patched:

  [+] Cryptographic signature on qemu-2.10.0.tar.xz checks out.
  [*] Uncompressing archive (this will take a while)...
  [+] Unpacking successful.
  [*] Configuring QEMU for ...
  [*] Applying patches...
  patching file linux-user/elfload.c
  patching file accel/tcg/cpu-exec.c
  patching file linux-user/syscall.c
  [+] Patching done.

EDIT:

when i normally build qemu support the patching looks more like this:

[*] Applying patches...
patching file linux-user/elfload.c
patching file accel/tcg/cpu-exec.c
patching file linux-user/syscall.c
patching file configure
Hunk #1 succeeded at 3855 (offset -68 lines).
patching file util/memfd.c
[+] Patching done.
spencerwuwu commented 3 years ago

I was facing the same problem. After installing lsb-release in the docker, the qemu scripts will apply the patch successfully.

SpaceMoehre commented 3 years ago

actually fixes it

sharkov63 commented 1 year ago

I have the same problem: memfd patch isn't applied. Attempting to install it in WSL 2 Ubuntu 22.04, with Python 3 virtual env