search

rgommers / pixi-dev-scipystack

Experimental out-of-tree dev setup for NumPy, SciPy and related projects with Pixi
Other
6 stars 1 forks source link

How can you compile for a specific Python version? #12

Open dhirschfeld opened 1 week ago

dhirschfeld commented 1 week ago

pixi run wheel builds a Python 3.12 wheel no problems. I need a Python 3.10 wheel though. How can I specify the Python version to build for?

I naively added a constraint in the pixi.toml:

[dependencies]
python = "3.10"

...but that gave an error about a missing header file:

❯ pixi run wheel
✨ Pixi task (wheel in default): python -m build -wnx -Cbuild-dir=build-whl && cp dist/*.whl ../../wheelhouse/
* Building wheel...
+ meson setup --reconfigure ~/code/github/scipy/pixi-dev-scipystack/scipy/scipy ~/code/github/scipy/pixi-dev-scipystack/scipy/scipy/build-whl -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --native-file=~/code/github/scipy/pixi-dev-scipystack/scipy/scipy/build-whl/meson-python-native-file.ini
The Meson build system
Version: 1.5.1
Source dir: ~/code/github/scipy/pixi-dev-scipystack/scipy/scipy
Build dir: ~/code/github/scipy/pixi-dev-scipystack/scipy/scipy/build-whl
Build type: native build
Project name: scipy
Project version: 1.15.0.dev0+git20241011.1adc8cb
C compiler for the host machine: ~/code/github/scipy/pixi-dev-scipystack/scipy/.pixi/envs/default/bin/x86_64-conda-linux-gnu-cc (gcc 12.4.0 "x86_64-conda-linux-gnu-cc (conda-forge gcc 12.4.0-0) 12.4.0")
C linker for the host machine: ~/code/github/scipy/pixi-dev-scipystack/scipy/.pixi/envs/default/bin/x86_64-conda-linux-gnu-cc ld.bfd 2.40
C++ compiler for the host machine: ~/code/github/scipy/pixi-dev-scipystack/scipy/.pixi/envs/default/bin/x86_64-conda-linux-gnu-c++ (gcc 12.4.0 "x86_64-conda-linux-gnu-c++ (conda-forge gcc 12.4.0-0) 12.4.0")
C++ linker for the host machine: ~/code/github/scipy/pixi-dev-scipystack/scipy/.pixi/envs/default/bin/x86_64-conda-linux-gnu-c++ ld.bfd 2.40
Cython compiler for the host machine: cython (cython 3.0.11)
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program python found: YES (~/code/github/scipy/pixi-dev-scipystack/scipy/.pixi/envs/default/bin/python)
Program cython found: YES (~/code/github/scipy/pixi-dev-scipystack/scipy/.pixi/envs/default/bin/cython)
Compiler for C supports arguments -Wno-unused-but-set-variable: YES (cached)
Compiler for C supports arguments -Wno-unused-function: YES (cached)
Compiler for C supports arguments -Wno-conversion: YES (cached)
Compiler for C supports arguments -Wno-misleading-indentation: YES (cached)
Library m found: YES
Fortran compiler for the host machine: ~/code/github/scipy/pixi-dev-scipystack/scipy/.pixi/envs/default/bin/x86_64-conda-linux-gnu-gfortran (gcc 12.4.0 "GNU Fortran (conda-forge gcc 12.4.0-0) 12.4.0")
Fortran linker for the host machine: ~/code/github/scipy/pixi-dev-scipystack/scipy/.pixi/envs/default/bin/x86_64-conda-linux-gnu-gfortran ld.bfd 2.40
Compiler for Fortran supports arguments -Wno-conversion: YES (cached)
Checking if "-Wl,--version-script" : links: YES (cached)
Program tools/generate_f2pymod.py found: YES (~/code/github/scipy/pixi-dev-scipystack/scipy/.pixi/envs/default/bin/python3.10 ~/code/github/scipy/pixi-dev-scipystack/scipy/scipy/tools/generate_f2pymod.py)
Program scipy/_build_utils/tempita.py found: YES (~/code/github/scipy/pixi-dev-scipystack/scipy/.pixi/envs/default/bin/python3.10 ~/code/github/scipy/pixi-dev-scipystack/scipy/scipy/scipy/_build_utils/tempita.py)
Program pythran found: YES 0.16.1 0.16.1 (~/code/github/scipy/pixi-dev-scipystack/scipy/.pixi/envs/default/bin/pythran)
Found pkg-config: YES (~/code/github/scipy/pixi-dev-scipystack/scipy/.pixi/envs/default/bin/pkg-config) 0.29.2
Did not find CMake 'cmake'
Found CMake: NO
Run-time dependency xsimd found: NO (tried pkgconfig and cmake)
Dependency threads found: YES unknown (cached)
Library npymath found: YES
Dependency pybind11 found: YES 2.13.6 (cached)
Program f2py found: YES (~/code/github/scipy/pixi-dev-scipystack/scipy/.pixi/envs/default/bin/f2py)
Run-time dependency scipy-openblas found: NO (tried pkgconfig)
Dependency openblas found: YES 0.3.27 (cached)
Dependency openblas found: YES 0.3.27 (cached)
Compiler for C supports arguments -Wno-maybe-uninitialized: YES (cached)
Compiler for C supports arguments -Wno-discarded-qualifiers: YES (cached)
Compiler for C supports arguments -Wno-empty-body: YES (cached)
Compiler for C supports arguments -Wno-implicit-function-declaration: YES (cached)
Compiler for C supports arguments -Wno-parentheses: YES (cached)
Compiler for C supports arguments -Wno-switch: YES (cached)
Compiler for C supports arguments -Wno-unused-label: YES (cached)
Compiler for C supports arguments -Wno-unused-result: YES (cached)
Compiler for C supports arguments -Wno-unused-variable: YES (cached)
Compiler for C++ supports arguments -Wno-bitwise-instead-of-logical: NO (cached)
Compiler for C++ supports arguments -Wno-cpp: YES (cached)
Compiler for C++ supports arguments -Wno-class-memaccess: YES (cached)
Compiler for C++ supports arguments -Wno-deprecated-declarations: YES (cached)
Compiler for C++ supports arguments -Wno-deprecated-builtins: NO (cached)
Compiler for C++ supports arguments -Wno-format-truncation: YES (cached)
Compiler for C++ supports arguments -Wno-non-virtual-dtor: YES (cached)
Compiler for C++ supports arguments -Wno-sign-compare: YES (cached)
Compiler for C++ supports arguments -Wno-switch: YES (cached)
Compiler for C++ supports arguments -Wno-terminate: YES (cached)
Compiler for C++ supports arguments -Wno-unused-but-set-variable: YES (cached)
Compiler for C++ supports arguments -Wno-unused-function: YES (cached)
Compiler for C++ supports arguments -Wno-unused-local-typedefs: YES (cached)
Compiler for C++ supports arguments -Wno-unused-variable: YES (cached)
Compiler for C++ supports arguments -Wno-int-in-bool-context: YES (cached)
Compiler for Fortran supports arguments -Wno-argument-mismatch: YES (cached)
Compiler for Fortran supports arguments -Wno-conversion: YES (cached)
Compiler for Fortran supports arguments -Wno-intrinsic-shadow: YES (cached)
Compiler for Fortran supports arguments -Wno-maybe-uninitialized: YES (cached)
Compiler for Fortran supports arguments -Wno-surprising: YES (cached)
Compiler for Fortran supports arguments -Wno-uninitialized: YES (cached)
Compiler for Fortran supports arguments -Wno-unused-dummy-argument: YES (cached)
Compiler for Fortran supports arguments -Wno-unused-label: YES (cached)
Compiler for Fortran supports arguments -Wno-unused-variable: YES (cached)
Compiler for Fortran supports arguments -Wno-tabs: YES (cached)
Compiler for Fortran supports arguments -Wno-argument-mismatch: YES (cached)
Compiler for Fortran supports arguments -Wno-conversion: YES (cached)
Compiler for Fortran supports arguments -Wno-maybe-uninitialized: YES (cached)
Compiler for Fortran supports arguments -Wno-unused-dummy-argument: YES (cached)
Compiler for Fortran supports arguments -Wno-unused-label: YES (cached)
Compiler for Fortran supports arguments -Wno-unused-variable: YES (cached)
Compiler for Fortran supports arguments -Wno-tabs: YES (cached)
Checking if "Check atomic builtins without -latomic" : links: YES (cached)
Configuring __config__.py using configuration
Checking for function "open_memstream" : YES (cached)
Configuring messagestream_config.h using configuration
Program _generate_pyx.py found: YES (~/code/github/scipy/pixi-dev-scipystack/scipy/.pixi/envs/default/bin/python3.10 ~/code/github/scipy/pixi-dev-scipystack/scipy/scipy/scipy/special/_generate_pyx.py)
Program _generate_pyx.py found: YES (~/code/github/scipy/pixi-dev-scipystack/scipy/.pixi/envs/default/bin/python3.10 ~/code/github/scipy/pixi-dev-scipystack/scipy/scipy/scipy/linalg/_generate_pyx.py)
Program ../_generate_sparsetools.py found: YES (~/code/github/scipy/pixi-dev-scipystack/scipy/.pixi/envs/default/bin/python3.10 ~/code/github/scipy/pixi-dev-scipystack/scipy/scipy/scipy/sparse/sparsetools/../_generate_sparsetools.py)
Checking for size of "void*" : 8 (cached)
Compiler for Fortran supports arguments -w: YES (cached)
Build targets in project: 194

scipy 1.15.0.dev0+git20241011.1adc8cb

  User defined options
    Native files: ~/code/github/scipy/pixi-dev-scipystack/scipy/scipy/build-whl/meson-python-native-file.ini
    buildtype   : release
    b_ndebug    : if-release
    b_vscrt     : md

Found ninja-1.12.1 at ~/code/github/scipy/pixi-dev-scipystack/scipy/.pixi/envs/default/bin/ninja
+ ~/code/github/scipy/pixi-dev-scipystack/scipy/.pixi/envs/default/bin/ninja
[9/1259] Compiling C object scipy/lib_fortranobject.a.p/.._.._.pixi_envs_default_lib_python3.10_site-packages_numpy_f2py_src_fortranobject.c.o
FAILED: scipy/lib_fortranobject.a.p/.._.._.pixi_envs_default_lib_python3.10_site-packages_numpy_f2py_src_fortranobject.c.o 
~/code/github/scipy/pixi-dev-scipystack/scipy/.pixi/envs/default/bin/x86_64-conda-linux-gnu-cc -Iscipy/lib_fortranobject.a.p -Iscipy -I../scipy -I../../.pixi/envs/default/lib/python3.10/site-packages/numpy/_core/include -I../../.pixi/envs/default/lib/python3.10/site-packages/numpy/f2py/src -I~/code/github/scipy/pixi-dev-scipystack/scipy/.pixi/envs/default/include/python3.10 -fvisibility=hidden -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -std=c17 -O3 -Wno-unused-but-set-variable -Wno-unused-function -Wno-conversion -Wno-misleading-indentation -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem ~/code/github/scipy/pixi-dev-scipystack/scipy/.pixi/envs/default/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem ~/code/github/scipy/pixi-dev-scipystack/scipy/.pixi/envs/default/include -fPIC -DNPY_NO_DEPRECATED_API=NPY_1_9_API_VERSION -MD -MQ scipy/lib_fortranobject.a.p/.._.._.pixi_envs_default_lib_python3.10_site-packages_numpy_f2py_src_fortranobject.c.o -MF scipy/lib_fortranobject.a.p/.._.._.pixi_envs_default_lib_python3.10_site-packages_numpy_f2py_src_fortranobject.c.o.d -o scipy/lib_fortranobject.a.p/.._.._.pixi_envs_default_lib_python3.10_site-packages_numpy_f2py_src_fortranobject.c.o -c ../../.pixi/envs/default/lib/python3.10/site-packages/numpy/f2py/src/fortranobject.c
In file included from ../../.pixi/envs/default/lib/python3.10/site-packages/numpy/f2py/src/fortranobject.h:7,
                 from ../../.pixi/envs/default/lib/python3.10/site-packages/numpy/f2py/src/fortranobject.c:2:
~/code/github/scipy/pixi-dev-scipystack/scipy/.pixi/envs/default/include/python3.10/Python.h:44:10: fatal error: crypt.h: No such file or directory
   44 | #include <crypt.h>
      |          ^~~~~~~~~
compilation terminated.
[10/1259] Compiling C object scipy/libdummy_g77_abi_wrappers.a.p/_build_utils_src_wrap_dummy_g77_abi.c.o
FAILED: scipy/libdummy_g77_abi_wrappers.a.p/_build_utils_src_wrap_dummy_g77_abi.c.o 
~/code/github/scipy/pixi-dev-scipystack/scipy/.pixi/envs/default/bin/x86_64-conda-linux-gnu-cc -Iscipy/libdummy_g77_abi_wrappers.a.p -Iscipy -I../scipy -I../../.pixi/envs/default/lib/python3.10/site-packages/numpy/_core/include -I~/code/github/scipy/pixi-dev-scipystack/scipy/.pixi/envs/default/include/python3.10 -I~/code/github/scipy/pixi-dev-scipystack/scipy/.pixi/envs/default/include -fvisibility=hidden -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -std=c17 -O3 -Wno-unused-but-set-variable -Wno-unused-function -Wno-conversion -Wno-misleading-indentation -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem ~/code/github/scipy/pixi-dev-scipystack/scipy/.pixi/envs/default/include -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem ~/code/github/scipy/pixi-dev-scipystack/scipy/.pixi/envs/default/include -fPIC -DNPY_NO_DEPRECATED_API=NPY_1_9_API_VERSION -MD -MQ scipy/libdummy_g77_abi_wrappers.a.p/_build_utils_src_wrap_dummy_g77_abi.c.o -MF scipy/libdummy_g77_abi_wrappers.a.p/_build_utils_src_wrap_dummy_g77_abi.c.o.d -o scipy/libdummy_g77_abi_wrappers.a.p/_build_utils_src_wrap_dummy_g77_abi.c.o -c ../scipy/_build_utils/src/wrap_dummy_g77_abi.c
In file included from ../../.pixi/envs/default/lib/python3.10/site-packages/numpy/_core/include/numpy/npy_common.h:5,
                 from ../scipy/_build_utils/src/npy_cblas.h:9,
                 from ../scipy/_build_utils/src/wrap_dummy_g77_abi.c:25:
~/code/github/scipy/pixi-dev-scipystack/scipy/.pixi/envs/default/include/python3.10/Python.h:44:10: fatal error: crypt.h: No such file or directory
   44 | #include <crypt.h>
      |          ^~~~~~~~~
compilation terminated.
[18/1259] Copying file scipy/special/_hyp0f1.pxd
ninja: build stopped: subcommand failed.

ERROR Backend subprocess exited when trying to invoke build_wheel

How can a build for a specific Python version?

rgommers commented 1 week ago

I naively added a constraint in the pixi.toml:

What you did should work, so that's probably an issue with a cached/non-empty build directory or some such thing.

More importantly though, the wheel produced by pixi run wheel aren't meant to be portable to other environments/machines, so it's unlikely that you actually need this kind of wheel. If that's what you want, it needs an auditwheel (or delocate or delvewheel) invocation in the command for the wheel pixi task.

dhirschfeld commented 6 days ago

What you did should work, so that's probably an issue with a cached/non-empty build directory or some such thing.

I did a git clean -xfd in both pixi-dev-scipystack and pixi-dev-scipystack/scipy/scipy and it still failed with the crypt.h error. I would've assumed a missing dependency. Does a 3.10 build work for you with pixi-dev-scipystack?

rgommers commented 5 days ago

Yes, that works fine. I've tested both 3.10.0 and 3.10.14, all good. I was thinking perhaps you got 3.10.0, and the correct constraint is one of 3.10.* or >=3.10,<3.11 - you can check with pixi ls python. Either way it all works for me.

dhirschfeld commented 5 days ago

Thanks - it's good to know that it should work! Now I just need to figure out what I'm doing wrong... 😬