trailofbits / pe-parse

Principled, lightweight C/C++ PE parser
MIT License
794 stars 155 forks source link

python setup.py sdist builds incorrect packages due to missing files in MANIFEST.in #183

Closed dirtyfilthy closed 1 year ago

dirtyfilthy commented 1 year ago

Currently "pip install pepy" doesn't work, because the necessary files aren't included in the MANIFEST.in to build the wheel.

Here is an example of attempting to build and install with setup.py sdist

`└─$ git clone https://github.com/trailofbits/pe-parse.git
Cloning into 'pe-parse'...
remote: Enumerating objects: 1700, done.
remote: Counting objects: 100% (263/263), done.
remote: Compressing objects: 100% (150/150), done.
remote: Total 1700 (delta 110), reused 215 (delta 89), pack-reused 1437
Receiving objects: 100% (1700/1700), 663.29 KiB | 1.47 MiB/s, done.
Resolving deltas: 100% (868/868), done.

┌──(alhazred㉿X774412)-[~/src]
└─$ cd pe-parse         

┌──(alhazred㉿X774412)-[~/src/pe-parse]
└─$ python3 setup.py sdist
running sdist
running egg_info
creating pepy.egg-info
writing pepy.egg-info/PKG-INFO
writing dependency_links to pepy.egg-info/dependency_links.txt
writing top-level names to pepy.egg-info/top_level.txt
writing manifest file 'pepy.egg-info/SOURCES.txt'
reading manifest file 'pepy.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
writing manifest file 'pepy.egg-info/SOURCES.txt'
running check
creating pepy-2.1.1
creating pepy-2.1.1/pe-parser-library
creating pepy-2.1.1/pe-parser-library/include
creating pepy-2.1.1/pe-parser-library/include/pe-parse
creating pepy-2.1.1/pepy
creating pepy-2.1.1/pepy.egg-info
copying files to pepy-2.1.1...
copying LICENSE -> pepy-2.1.1
copying MANIFEST.in -> pepy-2.1.1
copying README.md -> pepy-2.1.1
copying VERSION -> pepy-2.1.1
copying setup.py -> pepy-2.1.1
copying pe-parser-library/include/pe-parse/nt-headers.h -> pepy-2.1.1/pe-parser-library/include/pe-parse
copying pe-parser-library/include/pe-parse/parse.h -> pepy-2.1.1/pe-parser-library/include/pe-parse
copying pe-parser-library/include/pe-parse/to_string.h -> pepy-2.1.1/pe-parser-library/include/pe-parse
copying pepy/README.md -> pepy-2.1.1/pepy
copying pepy.egg-info/PKG-INFO -> pepy-2.1.1/pepy.egg-info
copying pepy.egg-info/SOURCES.txt -> pepy-2.1.1/pepy.egg-info
copying pepy.egg-info/dependency_links.txt -> pepy-2.1.1/pepy.egg-info
copying pepy.egg-info/top_level.txt -> pepy-2.1.1/pepy.egg-info
Writing pepy-2.1.1/setup.cfg
creating dist
Creating tar archive
removing 'pepy-2.1.1' (and everything under it)

┌──(alhazred㉿X774412)-[~/src/pe-parse]
└─$ pip install dist/pepy-2.1.1.tar.gz 
Defaulting to user installation because normal site-packages is not writeable
Processing ./dist/pepy-2.1.1.tar.gz
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: pepy
  Building wheel for pepy (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [15 lines of output]
      running bdist_wheel
      running build
      running build_ext
      building 'pepy' extension
      creating build
      creating build/temp.linux-x86_64-cpython-310
      creating build/temp.linux-x86_64-cpython-310/tmp
      creating build/temp.linux-x86_64-cpython-310/tmp/pip-req-build-kykyacsh
      creating build/temp.linux-x86_64-cpython-310/tmp/pip-req-build-kykyacsh/pe-parser-library
      creating build/temp.linux-x86_64-cpython-310/tmp/pip-req-build-kykyacsh/pe-parser-library/src
      creating build/temp.linux-x86_64-cpython-310/tmp/pip-req-build-kykyacsh/pepy
      x86_64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DPEPARSE_VERSION=\"2.1.1\" -I/usr/local/include -I/opt/local/include -I/usr/include -I/tmp/pip-req-build-kykyacsh/pe-parser-library/include -I/usr/include/python3.10 -c /tmp/pip-req-build-kykyacsh/pe-parser-library/src/buffer.cpp -o build/temp.linux-x86_64-cpython-310/tmp/pip-req-build-kykyacsh/pe-parser-library/src/buffer.o -std=c++17
      cc1plus: fatal error: /tmp/pip-req-build-kykyacsh/pe-parser-library/src/buffer.cpp: No such file or directory
      compilation terminated.
      error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pepy
  Running setup.py clean for pepy
Failed to build pepy
Installing collected packages: pepy
  Running setup.py install for pepy ... error
  error: subprocess-exited-with-error

  × Running setup.py install for pepy did not run successfully.
  │ exit code: 1
  ╰─> [17 lines of output]
      running install
      /usr/lib/python3/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        warnings.warn(
      running build
      running build_ext
      building 'pepy' extension
      creating build
      creating build/temp.linux-x86_64-cpython-310
      creating build/temp.linux-x86_64-cpython-310/tmp
      creating build/temp.linux-x86_64-cpython-310/tmp/pip-req-build-kykyacsh
      creating build/temp.linux-x86_64-cpython-310/tmp/pip-req-build-kykyacsh/pe-parser-library
      creating build/temp.linux-x86_64-cpython-310/tmp/pip-req-build-kykyacsh/pe-parser-library/src
      creating build/temp.linux-x86_64-cpython-310/tmp/pip-req-build-kykyacsh/pepy
      x86_64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DPEPARSE_VERSION=\"2.1.1\" -I/usr/local/include -I/opt/local/include -I/usr/include -I/tmp/pip-req-build-kykyacsh/pe-parser-library/include -I/usr/include/python3.10 -c /tmp/pip-req-build-kykyacsh/pe-parser-library/src/buffer.cpp -o build/temp.linux-x86_64-cpython-310/tmp/pip-req-build-kykyacsh/pe-parser-library/src/buffer.o -std=c++17
      cc1plus: fatal error: /tmp/pip-req-build-kykyacsh/pe-parser-library/src/buffer.cpp: No such file or directory
      compilation terminated.
      error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> pepy

note: This is an issue with the package mentioned above, not pip.
hint: See above for output from the failure.

`