search

plasma-umass / slipcover

Near Zero-Overhead Python Code Coverage
Apache License 2.0
475 stars 17 forks source link

Include pyptr.h in sdist, allow conda build config via CXX env vars. #31

Closed asford closed 1 year ago

asford commented 1 year ago

Minor fixes required for builds under conda.

lgtm-com[bot] commented 1 year ago

This pull request introduces 1 alert when merging b8c075d87e124c83ae8b9e1f3a27c330b54aa121 into 94a4d8235264629eec0e8b06134f0fbf95bdb324 - view on LGTM.com

new alerts:

jaltmayerpizzorno commented 1 year ago

Thank you for the PR!

asford commented 1 year ago

Roger, this has been updated to include what I needed to get a functional conda build on all platforms.

Notably, this pulls build configurations from CXX and CXXFLAGS to specify the target compiler and flags.

Added fallbacks so this should be a no-op in your wheel build environment, where maybe CXX isn't set to g++?

lgtm-com[bot] commented 1 year ago

This pull request introduces 1 alert when merging 76abd47ff9126cbb47d2c842de5b2205d0ed8dac into 94a4d8235264629eec0e8b06134f0fbf95bdb324 - view on LGTM.com

new alerts:

jaltmayerpizzorno commented 1 year ago

Sorry, it's still invoking it as gcc (from https://github.com/plasma-umass/slipcover/actions/runs/3420804089/jobs/5696101150):

building 'slipcover.probe' extension
creating build/temp.linux-x86_64-cpython-310
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/opt/_internal/cpython-3.10.8/include/python3.10 -c probe.cxx -o build/temp.linux-x86_64-cpython-310/probe.o -std=c++[17](https://github.com/plasma-umass/slipcover/actions/runs/3420804089/jobs/5696101150#step:10:18)
probe.cxx:21:10: error: ‘byte’ in namespace ‘std’ does not name a type
     std::byte* _code;
          ^~~~
probe.cxx: In constructor ‘Probe::Probe(PyObject*, PyObject*, PyObject*, PyObject*)’:
probe.cxx:29:61: error: class ‘Probe’ does not have any field named ‘_code’
         _d_miss_threshold(PyLong_AsLong(d_miss_threshold)), _code(nullptr) {}
                                                             ^~~~~
probe.cxx: In member function ‘PyObject* Probe::signal()’:
probe.cxx:45:29: error: ‘_code’ was not declared in this scope
         if (!_signalled || (_code == nullptr && _d_miss_threshold < -1)) {
                             ^~~~~
probe.cxx:69:17: error: ‘_code’ was not declared in this scope
[...]

I think setup.py's modules are reading CXX when they start up, so that changing later doesn't affect them.

There was also a missing include os, but I fixed it before the run above.

I've been thinking that I should switch the project to a src directory scheme; that would avoid the missing include issue for any future files, but I'm waiting for this PR to be finalized before doing that.

lgtm-com[bot] commented 1 year ago

This pull request introduces 1 alert when merging 69357f6858514fd62c2bbf540a1dc1e3912054c8 into 94a4d8235264629eec0e8b06134f0fbf95bdb324 - view on LGTM.com

new alerts:

asford commented 1 year ago

Thanks for catching the import os issue.

I'm still seeing this build failure when running your current main in the manylinux container, which is resolved by #33

asford commented 1 year ago

I've rebased this on #33, which fixes the build issue and allows us to remove the g++ override.

Verified via local container build:

alexford@AB-00RQ8221400C:~/asford/slipcover$ git fpush
Enumerating objects: 21, done.
Counting objects: 100% (21/21), done.
Delta compression using up to 8 threads
Compressing objects: 100% (18/18), done.
Writing objects: 100% (18/18), 2.26 KiB | 578.00 KiB/s, done.
Total 18 (delta 11), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (11/11), completed with 2 local objects.
To github.com:asford/slipcover.git
 + 69357f6...52f2909 sdist -> sdist (forced update)
alexford@AB-00RQ8221400C:~/asford/slipcover$ act -j test-build-wheel-manylinux workflow_dispatch
[tests/test-build-wheel-manylinux-1] 🚧  Skipping unsupported platform -- Try running with `-P ubuntu-latest=...`
[tests/test-build-wheel-manylinux-2] 🚧  Skipping unsupported platform -- Try running with `-P ubuntu-latest=...`
[tests/test-build-wheel-manylinux-3] 🚧  Skipping unsupported platform -- Try running with `-P ubuntu-latest=...`
[tests/test-build-wheel-manylinux-4] 🚀  Start image=quay.io/pypa/manylinux2014_x86_64
[tests/test-build-wheel-manylinux-4]   🐳  docker pull image=quay.io/pypa/manylinux2014_x86_64 platform= username= forcePull=false
[tests/test-build-wheel-manylinux-4]   🐳  docker create image=quay.io/pypa/manylinux2014_x86_64 platform= entrypoint=["/usr/bin/tail" "-f" "/dev/null"] cmd=[]
[tests/test-build-wheel-manylinux-4]   🐳  docker run image=quay.io/pypa/manylinux2014_x86_64 platform= entrypoint=["/usr/bin/tail" "-f" "/dev/null"] cmd=[]
[tests/test-build-wheel-manylinux-4] 🧪  Matrix: map[container:quay.io/pypa/manylinux2014_x86_64 os:ubuntu-latest]
[tests/test-build-wheel-manylinux-4] ⭐ Run Main actions/checkout@v2
[tests/test-build-wheel-manylinux-4]   🐳  docker cp src=/home/alexford/asford/slipcover/. dst=/home/alexford/asford/slipcover
[tests/test-build-wheel-manylinux-4]   ✅  Success - Main actions/checkout@v2
[tests/test-build-wheel-manylinux-4] ⭐ Run Main set up python
[tests/test-build-wheel-manylinux-4]   🐳  docker exec cmd=[sh -e -c /var/run/act/workflow/1.sh] user= workdir=
| /opt/python/cp310-cp310/bin
[tests/test-build-wheel-manylinux-4]   ✅  Success - Main set up python
[tests/test-build-wheel-manylinux-4] ⭐ Run Main install dependencies
[tests/test-build-wheel-manylinux-4]   🐳  docker exec cmd=[sh -e -c /var/run/act/workflow/2.sh] user= workdir=
| Requirement already satisfied: setuptools in /opt/_internal/cpython-3.10.8/lib/python3.10/site-packages (65.5.1)
| Requirement already satisfied: wheel in /opt/_internal/cpython-3.10.8/lib/python3.10/site-packages (0.38.2)
| Collecting twine
|   Downloading twine-4.0.1-py3-none-any.whl (36 kB)
| Collecting requests-toolbelt!=0.9.0,>=0.8.0
|   Downloading requests_toolbelt-0.10.1-py2.py3-none-any.whl (54 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.5/54.5 kB 9.2 MB/s eta 0:00:00
| Collecting urllib3>=1.26.0
|   Downloading urllib3-1.26.12-py2.py3-none-any.whl (140 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 140.4/140.4 kB 11.6 MB/s eta 0:00:00
| Collecting pkginfo>=1.8.1
|   Downloading pkginfo-1.8.3-py2.py3-none-any.whl (26 kB)
| Collecting rich>=12.0.0
|   Downloading rich-12.6.0-py3-none-any.whl (237 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 237.5/237.5 kB 9.0 MB/s eta 0:00:00
| Collecting importlib-metadata>=3.6
|   Downloading importlib_metadata-5.0.0-py3-none-any.whl (21 kB)
| Collecting requests>=2.20
|   Downloading requests-2.28.1-py3-none-any.whl (62 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.8/62.8 kB 12.3 MB/s eta 0:00:00
| Collecting rfc3986>=1.4.0
|   Downloading rfc3986-2.0.0-py2.py3-none-any.whl (31 kB)
| Collecting readme-renderer>=35.0
|   Downloading readme_renderer-37.3-py3-none-any.whl (14 kB)
| Collecting keyring>=15.1
|   Downloading keyring-23.11.0-py3-none-any.whl (36 kB)
| Collecting zipp>=0.5
|   Downloading zipp-3.10.0-py3-none-any.whl (6.2 kB)
| Collecting SecretStorage>=3.2
|   Downloading SecretStorage-3.3.3-py3-none-any.whl (15 kB)
| Collecting jeepney>=0.4.2
|   Downloading jeepney-0.8.0-py3-none-any.whl (48 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.4/48.4 kB 10.9 MB/s eta 0:00:00
| Collecting jaraco.classes
|   Downloading jaraco.classes-3.2.3-py3-none-any.whl (6.0 kB)
| Collecting bleach>=2.1.0
|   Downloading bleach-5.0.1-py3-none-any.whl (160 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 160.9/160.9 kB 17.5 MB/s eta 0:00:00
| Collecting docutils>=0.13.1
|   Downloading docutils-0.19-py3-none-any.whl (570 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 570.5/570.5 kB 6.8 MB/s eta 0:00:00
| Collecting Pygments>=2.5.1
|   Downloading Pygments-2.13.0-py3-none-any.whl (1.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 4.5 MB/s eta 0:00:00
| Collecting charset-normalizer<3,>=2
|   Downloading charset_normalizer-2.1.1-py3-none-any.whl (39 kB)
| Collecting idna<4,>=2.5
|   Downloading idna-3.4-py3-none-any.whl (61 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.5/61.5 kB 4.6 MB/s eta 0:00:00
| Collecting certifi>=2017.4.17
|   Downloading certifi-2022.9.24-py3-none-any.whl (161 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 161.1/161.1 kB 6.3 MB/s eta 0:00:00
| Collecting commonmark<0.10.0,>=0.9.0
|   Downloading commonmark-0.9.1-py2.py3-none-any.whl (51 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 51.1/51.1 kB 8.8 MB/s eta 0:00:00
| Collecting webencodings
|   Downloading webencodings-0.5.1-py2.py3-none-any.whl (11 kB)
| Collecting six>=1.9.0
|   Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
| Collecting cryptography>=2.0
|   Downloading cryptography-38.0.3-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.2/4.2 MB 7.3 MB/s eta 0:00:00
| Collecting more-itertools
|   Downloading more_itertools-9.0.0-py3-none-any.whl (52 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 52.8/52.8 kB 14.2 MB/s eta 0:00:00
| Collecting cffi>=1.12
|   Downloading cffi-1.15.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (441 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 441.8/441.8 kB 11.1 MB/s eta 0:00:00
| Collecting pycparser
|   Downloading pycparser-2.21-py2.py3-none-any.whl (118 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 118.7/118.7 kB 16.3 MB/s eta 0:00:00
| Installing collected packages: webencodings, commonmark, zipp, urllib3, six, rfc3986, Pygments, pycparser, pkginfo, more-itertools, jeepney, idna, docutils, charset-normalizer, certifi, rich, requests, jaraco.classes, importlib-metadata, cffi, bleach, requests-toolbelt, readme-renderer, cryptography, SecretStorage, keyring, twine
| Successfully installed Pygments-2.13.0 SecretStorage-3.3.3 bleach-5.0.1 certifi-2022.9.24 cffi-1.15.1 charset-normalizer-2.1.1 commonmark-0.9.1 cryptography-38.0.3 docutils-0.19 idna-3.4 importlib-metadata-5.0.0 jaraco.classes-3.2.3 jeepney-0.8.0 keyring-23.11.0 more-itertools-9.0.0 pkginfo-1.8.3 pycparser-2.21 readme-renderer-37.3 requests-2.28.1 requests-toolbelt-0.10.1 rfc3986-2.0.0 rich-12.6.0 six-1.16.0 twine-4.0.1 urllib3-1.26.12 webencodings-0.5.1 zipp-3.10.0
| WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
[tests/test-build-wheel-manylinux-4]   ✅  Success - Main install dependencies
[tests/test-build-wheel-manylinux-4] ⭐ Run Main build wheel
[tests/test-build-wheel-manylinux-4]   🐳  docker exec cmd=[sh -e -c /var/run/act/workflow/3.sh] user= workdir=
| running bdist_wheel
| running build
| running build_py
| creating build
| creating build/lib.linux-x86_64-cpython-310
| creating build/lib.linux-x86_64-cpython-310/slipcover
| copying slipcover/__main__.py -> build/lib.linux-x86_64-cpython-310/slipcover
| copying slipcover/bytecode.py -> build/lib.linux-x86_64-cpython-310/slipcover
| copying slipcover/slipcover.py -> build/lib.linux-x86_64-cpython-310/slipcover
| copying slipcover/branch.py -> build/lib.linux-x86_64-cpython-310/slipcover
| copying slipcover/__init__.py -> build/lib.linux-x86_64-cpython-310/slipcover
| running build_ext
| building 'slipcover.probe' extension
| creating build/temp.linux-x86_64-cpython-310
| gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/opt/_internal/cpython-3.10.8/include/python3.10 -c probe.cxx -o build/temp.linux-x86_64-cpython-310/probe.o -std=c++17
| g++ -pthread -shared build/temp.linux-x86_64-cpython-310/probe.o -o build/lib.linux-x86_64-cpython-310/slipcover/probe.cpython-310-x86_64-linux-gnu.so
| /opt/python/cp310-cp310/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
|   warnings.warn(
| installing to build/bdist.linux-x86_64/wheel
| running install
| running install_lib
| creating build/bdist.linux-x86_64
| creating build/bdist.linux-x86_64/wheel
| creating build/bdist.linux-x86_64/wheel/slipcover
| copying build/lib.linux-x86_64-cpython-310/slipcover/__main__.py -> build/bdist.linux-x86_64/wheel/slipcover
| copying build/lib.linux-x86_64-cpython-310/slipcover/bytecode.py -> build/bdist.linux-x86_64/wheel/slipcover
| copying build/lib.linux-x86_64-cpython-310/slipcover/slipcover.py -> build/bdist.linux-x86_64/wheel/slipcover
| copying build/lib.linux-x86_64-cpython-310/slipcover/probe.cpython-310-x86_64-linux-gnu.so -> build/bdist.linux-x86_64/wheel/slipcover
| copying build/lib.linux-x86_64-cpython-310/slipcover/branch.py -> build/bdist.linux-x86_64/wheel/slipcover
| copying build/lib.linux-x86_64-cpython-310/slipcover/__init__.py -> build/bdist.linux-x86_64/wheel/slipcover
| running install_egg_info
| running egg_info
| creating slipcover.egg-info
| writing slipcover.egg-info/PKG-INFO
| writing dependency_links to slipcover.egg-info/dependency_links.txt
| writing requirements to slipcover.egg-info/requires.txt
| writing top-level names to slipcover.egg-info/top_level.txt
| writing manifest file 'slipcover.egg-info/SOURCES.txt'
| reading manifest file 'slipcover.egg-info/SOURCES.txt'
| reading manifest template 'MANIFEST.in'
| warning: no files found matching 'dev-build.txt'
| adding license file 'LICENSE'
| writing manifest file 'slipcover.egg-info/SOURCES.txt'
| Copying slipcover.egg-info to build/bdist.linux-x86_64/wheel/slipcover-0.2.0-py3.10.egg-info
| running install_scripts
| creating build/bdist.linux-x86_64/wheel/slipcover-0.2.0.dist-info/WHEEL
| creating 'dist/slipcover-0.2.0-cp310-cp310-linux_x86_64.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
| adding 'slipcover/__init__.py'
| adding 'slipcover/__main__.py'
| adding 'slipcover/branch.py'
| adding 'slipcover/bytecode.py'
| adding 'slipcover/probe.cpython-310-x86_64-linux-gnu.so'
| adding 'slipcover/slipcover.py'
| adding 'slipcover-0.2.0.dist-info/LICENSE'
| adding 'slipcover-0.2.0.dist-info/METADATA'
| adding 'slipcover-0.2.0.dist-info/WHEEL'
| adding 'slipcover-0.2.0.dist-info/top_level.txt'
| adding 'slipcover-0.2.0.dist-info/RECORD'
| removing build/bdist.linux-x86_64/wheel
[tests/test-build-wheel-manylinux-4]   ✅  Success - Main build wheel
[tests/test-build-wheel-manylinux-4] ⭐ Run Main run auditwheel for manylinux
[tests/test-build-wheel-manylinux-4]   🐳  docker exec cmd=[sh -e -c /var/run/act/workflow/4.sh] user= workdir=
| INFO:auditwheel.main_repair:Repairing slipcover-0.2.0-cp310-cp310-linux_x86_64.whl
| INFO:auditwheel.main_repair:Wheel is eligible for a higher priority tag. You requested manylinux2014_x86_64 but I have found this wheel is eligible for manylinux_2_5_x86_64.
| INFO:auditwheel.wheeltools:Previous filename tags: linux_x86_64
| INFO:auditwheel.wheeltools:New filename tags: manylinux_2_5_x86_64, manylinux1_x86_64, manylinux_2_17_x86_64, manylinux2014_x86_64
| INFO:auditwheel.wheeltools:Previous WHEEL info tags: cp310-cp310-linux_x86_64
| INFO:auditwheel.wheeltools:New WHEEL info tags: cp310-cp310-manylinux_2_5_x86_64, cp310-cp310-manylinux1_x86_64, cp310-cp310-manylinux_2_17_x86_64, cp310-cp310-manylinux2014_x86_64
| INFO:auditwheel.main_repair:
| Fixed-up wheel written to /home/alexford/asford/slipcover/wheelhouse/slipcover-0.2.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
[tests/test-build-wheel-manylinux-4]   ✅  Success - Main run auditwheel for manylinux
[tests/test-build-wheel-manylinux-4] 🏁  Job succeeded
lgtm-com[bot] commented 1 year ago

This pull request introduces 1 alert when merging 52f2909b040d685e14194ce44b9ad36377bf0361 into 94a4d8235264629eec0e8b06134f0fbf95bdb324 - view on LGTM.com

new alerts: