numeristical / splinecalib

Location for the splinecalib package
MIT License
3 stars 3 forks source link

unable to install splinecalib #8

Open SimpleCoder35 opened 2 weeks ago

SimpleCoder35 commented 2 weeks ago

I am unable to install splinecalib. There was a major update to numPy yesterday, and the package is attempting to install it as it is specified in the pyproject.toml

[build-system] requires = ["setuptools", "setuptools-scm", "numpy"] build-backend = "setuptools.build_meta"

!pip install splinecalib --verbose --no-deps

Using pip 24.0 from /usr/local/lib/python3.10/dist-packages/pip (python 3.10) Collecting splinecalib Downloading splinecalib-0.0.7.tar.gz (1.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 10.6 MB/s eta 0:00:00 Running command pip subprocess to install build dependencies Using pip 24.0 from /usr/local/lib/python3.10/dist-packages/pip (python 3.10) Collecting setuptools Obtaining dependency information for setuptools from https://files.pythonhosted.org/packages/de/88/70c5767a0e43eb4451c2200f07d042a4bcd7639276003a9c54a68cfcc1f8/setuptools-70.0.0-py3-none-any.whl.metadata Using cached setuptools-70.0.0-py3-none-any.whl.metadata (5.9 kB) Collecting setuptools-scm Obtaining dependency information for setuptools-scm from https://files.pythonhosted.org/packages/a0/b9/1906bfeb30f2fc13bb39bf7ddb8749784c05faadbd18a21cf141ba37bff2/setuptools_scm-8.1.0-py3-none-any.whl.metadata Downloading setuptools_scm-8.1.0-py3-none-any.whl.metadata (6.6 kB) Collecting numpy Obtaining dependency information for numpy from https://files.pythonhosted.org/packages/d6/a8/6a2419c40c7b6f7cb4ef52c532c88e55490c4fa92885964757d507adddce/numpy-2.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata Downloading numpy-2.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (60 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.9/60.9 kB 2.7 MB/s eta 0:00:00 Collecting packaging>=20 (from setuptools-scm) Obtaining dependency information for packaging>=20 from https://files.pythonhosted.org/packages/08/aa/cc0199a5f0ad350994d660967a8efb233fe0416e4639146c089643407ce6/packaging-24.1-py3-none-any.whl.metadata Using cached packaging-24.1-py3-none-any.whl.metadata (3.2 kB) Collecting tomli>=1 (from setuptools-scm) Obtaining dependency information for tomli>=1 from https://files.pythonhosted.org/packages/97/75/10a9ebee3fd790d20926a90a2547f0bf78f371b2f13aa822c759680ca7b9/tomli-2.0.1-py3-none-any.whl.metadata Using cached tomli-2.0.1-py3-none-any.whl.metadata (8.9 kB) Using cached setuptools-70.0.0-py3-none-any.whl (863 kB) Downloading setuptools_scm-8.1.0-py3-none-any.whl (43 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 43.7/43.7 kB 3.1 MB/s eta 0:00:00 Downloading numpy-2.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (19.3 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 19.3/19.3 MB 74.5 MB/s eta 0:00:00 Using cached packaging-24.1-py3-none-any.whl (53 kB) Using cached tomli-2.0.1-py3-none-any.whl (12 kB) Installing collected packages: tomli, setuptools, packaging, numpy, setuptools-scm Creating /tmp/pip-build-env-13p0zqnj/overlay/local/bin changing mode of /tmp/pip-build-env-13p0zqnj/overlay/local/bin/f2py to 755 changing mode of /tmp/pip-build-env-13p0zqnj/overlay/local/bin/numpy-config to 755 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. ipython 7.34.0 requires jedi>=0.16, which is not installed. astropy 5.3.4 requires numpy<2,>=1.21, but you have numpy 2.0.0 which is incompatible. cudf-cu12 24.4.1 requires numpy<2.0a0,>=1.23, but you have numpy 2.0.0 which is incompatible. cupy-cuda12x 12.2.0 requires numpy<1.27,>=1.20, but you have numpy 2.0.0 which is incompatible. ibis-framework 8.0.0 requires numpy<2,>=1, but you have numpy 2.0.0 which is incompatible. numba 0.58.1 requires numpy<1.27,>=1.22, but you have numpy 2.0.0 which is incompatible. rmm-cu12 24.4.0 requires numpy<2.0a0,>=1.23, but you have numpy 2.0.0 which is incompatible. scipy 1.11.4 requires numpy<1.28.0,>=1.21.6, but you have numpy 2.0.0 which is incompatible. tensorflow 2.15.0 requires numpy<2.0.0,>=1.23.5, but you have numpy 2.0.0 which is incompatible. Successfully installed numpy-2.0.0 packaging-24.1 setuptools-70.0.0 setuptools-scm-8.1.0 tomli-2.0.1 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv Installing build dependencies ... done Running command Getting requirements to build wheel running egg_info writing splinecalib.egg-info/PKG-INFO writing dependency_links to splinecalib.egg-info/dependency_links.txt writing requirements to splinecalib.egg-info/requires.txt writing top-level names to splinecalib.egg-info/top_level.txt ERROR setuptools_scm._file_finders.git listing git files failed - pretending there aren't any reading manifest file 'splinecalib.egg-info/SOURCES.txt' adding license file 'LICENSE' writing manifest file 'splinecalib.egg-info/SOURCES.txt' Getting requirements to build wheel ... done Running command pip subprocess to install backend dependencies Using pip 24.0 from /usr/local/lib/python3.10/dist-packages/pip (python 3.10) Collecting wheel Obtaining dependency information for wheel from https://files.pythonhosted.org/packages/7d/cd/d7460c9a869b16c3dd4e1e403cce337df165368c71d6af229a74699622ce/wheel-0.43.0-py3-none-any.whl.metadata Using cached wheel-0.43.0-py3-none-any.whl.metadata (2.2 kB) Using cached wheel-0.43.0-py3-none-any.whl (65 kB) Installing collected packages: wheel Creating /tmp/pip-build-env-13p0zqnj/normal/local/bin changing mode of /tmp/pip-build-env-13p0zqnj/normal/local/bin/wheel to 755 Successfully installed wheel-0.43.0 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv Installing backend dependencies ... done Running command Preparing metadata (pyproject.toml) running dist_info creating /tmp/pip-modern-metadata-n6nkp1n9/splinecalib.egg-info writing /tmp/pip-modern-metadata-n6nkp1n9/splinecalib.egg-info/PKG-INFO writing dependency_links to /tmp/pip-modern-metadata-n6nkp1n9/splinecalib.egg-info/dependency_links.txt writing requirements to /tmp/pip-modern-metadata-n6nkp1n9/splinecalib.egg-info/requires.txt writing top-level names to /tmp/pip-modern-metadata-n6nkp1n9/splinecalib.egg-info/top_level.txt writing manifest file '/tmp/pip-modern-metadata-n6nkp1n9/splinecalib.egg-info/SOURCES.txt' ERROR setuptools_scm._file_finders.git listing git files failed - pretending there aren't any reading manifest file '/tmp/pip-modern-metadata-n6nkp1n9/splinecalib.egg-info/SOURCES.txt' adding license file 'LICENSE' writing manifest file '/tmp/pip-modern-metadata-n6nkp1n9/splinecalib.egg-info/SOURCES.txt' creating '/tmp/pip-modern-metadata-n6nkp1n9/splinecalib-0.0.7.dist-info' Preparing metadata (pyproject.toml) ... done Building wheels for collected packages: splinecalib Running command Building wheel for splinecalib (pyproject.toml) running bdist_wheel running build running build_py creating build creating build/lib.linux-x86_64-cpython-310 creating build/lib.linux-x86_64-cpython-310/splinecalib copying splinecalib/init.py -> build/lib.linux-x86_64-cpython-310/splinecalib copying splinecalib/calib_utils.py -> build/lib.linux-x86_64-cpython-310/splinecalib copying splinecalib/splinecalib.py -> build/lib.linux-x86_64-cpython-310/splinecalib running build_ext building 'loss_fun_c' extension creating build/temp.linux-x86_64-cpython-310 creating build/temp.linux-x86_64-cpython-310/splinecalib x86_64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/tmp/pip-build-env-13p0zqnj/overlay/local/lib/python3.10/dist-packages/numpy/_core/include -I/usr/include/python3.10 -c splinecalib/loss_fun_c.c -o build/temp.linux-x86_64-cpython-310/splinecalib/loss_fun_c.o In file included from /tmp/pip-build-env-13p0zqnj/overlay/local/lib/python3.10/dist-packages/numpy/_core/include/numpy/ndarraytypes.h:1909, from /tmp/pip-build-env-13p0zqnj/overlay/local/lib/python3.10/dist-packages/numpy/_core/include/numpy/ndarrayobject.h:12, from /tmp/pip-build-env-13p0zqnj/overlay/local/lib/python3.10/dist-packages/numpy/_core/include/numpy/arrayobject.h:5, from splinecalib/loss_fun_c.c:1154: /tmp/pip-build-env-13p0zqnj/overlay/local/lib/python3.10/dist-packages/numpy/_core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp] 17 | #warning "Using deprecated NumPy API, disable it with " \ | ^~~ In file included from /usr/include/python3.10/Python.h:74, from splinecalib/loss_fun_c.c:16: splinecalib/loss_fun_c.c: In function ‘pyx_f_5numpy_PyDataType_SHAPE’: splinecalib/loss_fun_c.c:3539:39: error: ‘PyArray_Descr’ {aka ‘struct _PyArray_Descr’} has no member named ‘subarray’ 3539 | __Pyx_INCREF(((PyObject*)pyx_v_d->subarray->shape)); | ^~ /usr/include/python3.10/object.h:112:41: note: in definition of macro ‘_PyObject_CAST’ 112 | #define _PyObject_CAST(op) ((PyObject)(op)) | ^~ splinecalib/loss_fun_c.c:1796:27: note: in expansion of macro ‘Py_INCREF’ 1796 | #define Pyx_INCREF(r) Py_INCREF(r) | ^~~~~ splinecalib/loss_fun_c.c:3539:5: note: in expansion of macro ‘Pyx_INCREF’ 3539 | __Pyx_INCREF(((PyObject)pyx_v_d->subarray->shape)); | ^~~~ splinecalib/loss_fun_c.c:3540:36: error: ‘PyArray_Descr’ {aka ‘struct _PyArray_Descr’} has no member named ‘subarray’ 3540 | __pyx_r = ((PyObject*)pyx_v_d->subarray->shape); | ^~ error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1 error: subprocess-exited-with-error

× Building wheel for splinecalib (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip. full command: /usr/bin/python3 /usr/local/lib/python3.10/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py build_wheel /tmp/tmplmwqmkp1 cwd: /tmp/pip-install-ncs4wl02/splinecalib_cede067df01b4261b1685661973e52d3 Building wheel for splinecalib (pyproject.toml) ... error ERROR: Failed building wheel for splinecalib Failed to build splinecalib ERROR: Could not build wheels for splinecalib, which is required to install pyproject.toml-based projects

blucena commented 2 weeks ago

@SimpleCoder35 Thanks for alerting me to this. From what I can tell, the problem lies in finding the right version of numpy to fit not just splinecalib, but all the other packages you already have installed. A couple of ideas to try:

1) Try installing a specific version of numpy first. Based on the error message you might try a version >=1.23.5 but <2. That should be compatible with everything. Then when you install splinecalib, it will see the version of numpy you already have, and not attempt to install a new one.

2) Another thing you might try is installing without the --no-deps flag. Just pip install splinecalib. I'm not sure how the dependency resolver in pip works when you have the --no-deps flag, so that might be causing problems as well.

Let me know if this helps you get it installed.

SimpleCoder35 commented 2 weeks ago

@blucena , Thanks for the response.

I was trying without any dependencies, This is what I am trying

!pip install splinecalib --verbose --no-deps

but for some reason looks like pip is collecting the numpy 2.0 version.(from the log)

I have Numpy already installed(1.24.4), I froze the environment exactly to the previous working version. Still unable to install the package.

SimpleCoder35 commented 2 weeks ago

@blucena , update: I was able to install this package by modifying the setup.py added <2.0

install_requires=[
    "numpy>=1.16,<2.0",
    "scipy>=1.3"],
dcturner45 commented 2 weeks ago

I am seeing this issue as well. On macOS Sonoma 14.5 (if that matters), in a fresh conda environment with Python 3.9.19:

shell> conda create -y --name sctest python=3.9
shell> conda activate sctest
(sctest) shell> python --version
Python 3.9.19
(sctest) shell> pip install numpy==1.26.4 splinecalib
...
      splinecalib/loss_fun_c.c:3539:41: error: no member named 'subarray' in 'struct _PyArray_Descr'
          __Pyx_INCREF(((PyObject*)__pyx_v_d->subarray->shape));
                                   ~~~~~~~~~  ^
      splinecalib/loss_fun_c.c:1796:37: note: expanded from macro '__Pyx_INCREF'
        #define __Pyx_INCREF(r) Py_INCREF(r)
                                          ^
      /opt/homebrew/Caskroom/miniconda/base/envs/sctest/include/python3.9/object.h:411:49: note: expanded from macro 'Py_INCREF'
      #define Py_INCREF(op) _Py_INCREF(_PyObject_CAST(op))
                                                      ^~
      /opt/homebrew/Caskroom/miniconda/base/envs/sctest/include/python3.9/object.h:112:41: note: expanded from macro '_PyObject_CAST'
      #define _PyObject_CAST(op) ((PyObject*)(op))
                                              ^~
      splinecalib/loss_fun_c.c:3540:38: error: no member named 'subarray' in 'struct _PyArray_Descr'
          __pyx_r = ((PyObject*)__pyx_v_d->subarray->shape);
                                ~~~~~~~~~  ^
      1 warning and 2 errors generated.
      error: command '/usr/bin/clang' failed with exit code 1
      [end of output]

If explicitly setting an upper bound on numpy in install_requires fixes the issue (as mentioned here by @SimpleCoder35), is setuptools somehow pulling a different version of numpy than the one that's installed in the environment?

blucena commented 2 weeks ago

@dcturner45 Thanks for posting this. Not sure right now whether this is the same issue that @SimpleCoder35 faced or not. Yours seems to be related to the C-compilation in the Cython code and not necessarily the new version of numpy (esp since you pin the old version of numpy)

Let me do some investigation now and see if I can find a solution

Jennie-Myhren commented 2 weeks ago

I've also been seeing this issue (running a docker container w/ a Python v3.12 image). I've tried pinning the numpy version and setting an upper bound (project uses pyproject.toml, but I've also tried the solution above with install_requires in a setup.py file and tried installing numpy and/or splinecalib directly in the Dockerfile.)

creating build/temp.linux-x86_64-cpython-312/splinecalib
#13 307.8   gcc -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -I/tmp/tmpl3xgatdv/.venv/lib/python3.12/site-packages/numpy/_core/include -I/tmp/tmpl3xgatdv/.venv/include -I/usr/local/include/python3.12 -c splinecalib/loss_fun_c.c -o build/temp.linux-x86_64-cpython-312/splinecalib/loss_fun_c.o
#13 307.8   In file included from /tmp/tmpl3xgatdv/.venv/lib/python3.12/site-packages/numpy/_core/include/numpy/ndarraytypes.h:1909,
#13 307.8                    from /tmp/tmpl3xgatdv/.venv/lib/python3.12/site-packages/numpy/_core/include/numpy/ndarrayobject.h:12,
#13 307.8                    from /tmp/tmpl3xgatdv/.venv/lib/python3.12/site-packages/numpy/_core/include/numpy/arrayobject.h:5,
#13 307.8                    from splinecalib/loss_fun_c.c:1154:
#13 307.8   /tmp/tmpl3xgatdv/.venv/lib/python3.12/site-packages/numpy/_core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
#13 307.8      17 | #warning "Using deprecated NumPy API, disable it with " \
#13 307.8         |  ^~~~~~~
#13 307.8   In file included from /usr/local/include/python3.12/Python.h:38,
#13 307.8                    from splinecalib/loss_fun_c.c:16:
#13 307.8   splinecalib/loss_fun_c.c: In function ‘__pyx_f_5numpy_PyDataType_SHAPE’:
#13 307.8   splinecalib/loss_fun_c.c:3539:39: error: ‘PyArray_Descr’ {aka ‘struct _PyArray_Descr’} has no member named ‘subarray’
#13 307.8    3539 |     __Pyx_INCREF(((PyObject*)__pyx_v_d->subarray->shape));
#13 307.8         |                                       ^~
#13 307.8   /usr/local/include/python3.12/pyport.h:24:38: note: in definition of macro ‘_Py_CAST’
#13 307.8      24 | #define _Py_CAST(type, expr) ((type)(expr))
#13 307.8         |                                      ^~~~
#13 307.8   /usr/local/include/python3.12/object.h:661:35: note: in expansion of macro ‘_PyObject_CAST’
#13 307.8     661 | #  define Py_INCREF(op) Py_INCREF(_PyObject_CAST(op))
#13 307.8         |                                   ^~~~~~~~~~~~~~
#13 307.8   splinecalib/loss_fun_c.c:1796:27: note: in expansion of macro ‘Py_INCREF’
#13 307.8    1796 |   #define __Pyx_INCREF(r) Py_INCREF(r)
#13 307.8         |                           ^~~~~~~~~
#13 307.8   splinecalib/loss_fun_c.c:3539:5: note: in expansion of macro ‘__Pyx_INCREF’
#13 307.8    3539 |     __Pyx_INCREF(((PyObject*)__pyx_v_d->subarray->shape));
#13 307.8         |     ^~~~~~~~~~~~
#13 307.8   splinecalib/loss_fun_c.c:3540:36: error: ‘PyArray_Descr’ {aka ‘struct _PyArray_Descr’} has no member named ‘subarray’
#13 307.8    3540 |     __pyx_r = ((PyObject*)__pyx_v_d->subarray->shape);
#13 307.8         |                                    ^~
#13 307.8   error: command '/usr/bin/gcc' failed with exit code 1
#13 307.8   
#13 307.8 
#13 307.8   at /app/poetry/venv/lib/python3.12/site-packages/poetry/installation/chef.py:164 in _prepare
#13 307.8       160│ 
#13 307.8       161│                 error = ChefBuildError("\n\n".join(message_parts))
#13 307.8       162│ 
#13 307.8       163│             if error is not None:
#13 307.8     → 164│                 raise error from None
#13 307.8       165│ 
#13 307.8       166│             return path
#13 307.8       167│ 
#13 307.8       168│     def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:
#13 307.8 
#13 307.8 Note: This error originates from the build backend, and is likely not a problem with poetry but with splinecalib (0.0.7) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "splinecalib (==0.0.7)"'.
blucena commented 2 weeks ago

Thanks @Jennie-Myhren.

On my end I've started investigating and have found the following. I have two Macs, one older (2019) and one newer (Mac M2). On both I tried creating a new 3.9.19 environment (with pyenv) and installing splinecalib. In both cases it installed numpy 2.0.0 and scipy 1.3.1. On the older Mac everything installed fine. On the newer Mac I got the same error as @dcturner45 .

As far as I can tell, the problem is due to some issue with Cython's interface with Numpy. Cython creates C code that interacts with numpy and on some configurations it is not working. May or may not be related to the "#warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]".

I'll kepp digging. Please continue to comment with any relevant findings.

dcturner45 commented 2 weeks ago

@blucena When attempting to build from source locally in a fresh environment + pip install numpy==1.26.4, I get the same failure as from my earlier comment. However, pinning numpy == 1.26.4 in pyproject.toml allows the build to successfully complete. Not sure if pyproject is somehow pulling/caching its own numpy independent of the version that's installed in the environment?

The following is on an AWS EC2 instance running Amazon Linux (python3/pip3 are used because my EC2 shell aliases python/pip to the system python and the conda environment doesn't overwrite them):

[computer@ip ~]# conda create -y --name sctest python=3.9
[computer@ip ~]# conda activate sctest
(sctest) [computer@ip ~]# python3 --version
Python 3.9.19
(sctest) [computer@ip ~]# pip3 --version
pip 24.0 from /opt/conda/envs/sctest/lib/python3.9/site-packages/pip (python 3.9)
(sctest) [computer@ip ~]# git clone https://github.com/numeristical/splinecalib.git
(sctest) [computer@ip ~]# cd splinecalib/
(sctest) [computer@ip splinecalib]# pip3 install numpy==1.26.4
...
Installing collected packages: numpy
Successfully installed numpy-1.26.4

(sctest) [computer@ip splinecalib]# pip3 install .
...
      /mnt/tmp/pip-build-env-vzi0ze44/overlay/lib/python3.9/site-packages/numpy/_core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
       #warning "Using deprecated NumPy API, disable it with " \
        ^~~~~~~
      In file included from /opt/conda/envs/sctest/include/python3.9/pytime.h:6:0,
                       from /opt/conda/envs/sctest/include/python3.9/Python.h:81,
                       from splinecalib/loss_fun_c.c:16:
      splinecalib/loss_fun_c.c: In function ‘__pyx_f_5numpy_PyDataType_SHAPE’:
      splinecalib/loss_fun_c.c:3539:39: error: ‘PyArray_Descr {aka struct _PyArray_Descr}’ has no member named ‘subarray’
           __Pyx_INCREF(((PyObject*)__pyx_v_d->subarray->shape));
                                             ^
      /opt/conda/envs/sctest/include/python3.9/object.h:112:41: note: in definition of macro ‘_PyObject_CAST’
       #define _PyObject_CAST(op) ((PyObject*)(op))
                                               ^~
      splinecalib/loss_fun_c.c:1796:27: note: in expansion of macro ‘Py_INCREF’
         #define __Pyx_INCREF(r) Py_INCREF(r)
                                 ^~~~~~~~~
      splinecalib/loss_fun_c.c:3539:5: note: in expansion of macro ‘__Pyx_INCREF’
           __Pyx_INCREF(((PyObject*)__pyx_v_d->subarray->shape));
           ^~~~~~~~~~~~
      splinecalib/loss_fun_c.c:3540:36: error: ‘PyArray_Descr {aka struct _PyArray_Descr}’ has no member named ‘subarray’
           __pyx_r = ((PyObject*)__pyx_v_d->subarray->shape);
                                          ^~
      error: command '/bin/gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for splinecalib
Failed to build splinecalib
ERROR: Could not build wheels for splinecalib, which is required to install pyproject.toml-based projects

After pinning numpy == 1.26.4 in pyproject.toml:

(sctest) [computer@ip splinecalib]# cat pyproject.toml
[build-system]
requires = ["setuptools", "setuptools-scm", "numpy == 1.26.4"]
build-backend = "setuptools.build_meta"

(sctest) [computer@ip splinecalib]# pip install .
...
Requirement already satisfied: numpy>=1.16 in /opt/conda/envs/sctest/lib/python3.9/site-packages (from splinecalib==0.0.7) (1.26.4)
...
Building wheels for collected packages: splinecalib
  Building wheel for splinecalib (pyproject.toml) ... done
  Created wheel for splinecalib: filename=splinecalib-0.0.7-cp39-cp39-linux_x86_64.whl size=50883 sha256=dad4e860f00466bebfed736432ba7bcc1ae441c4ae40a2229b964d1433f1a0fe
  Stored in directory: /mnt/tmp/pip-ephem-wheel-cache-54fgf97z/wheels/41/91/3f/444f0a9bbacae2fdcca70969469ff7bac4cf22f94478188029
Successfully built splinecalib
Installing collected packages: scipy, splinecalib
Successfully installed scipy-1.13.1 splinecalib-0.0.7

Identical behavior is observed on my Mac (M1).

blucena commented 2 weeks ago

Thanks @dcturner45 . I also replicated the same behavior you show. I am mystified as to what is going on. I'm sure I have some weirdness in how my package is configured. For example, I notice that I am using both setup.py and pyproject.toml to specify requirements, which is probably wrong.

In the meantime, I have changed pyproject.toml to read "numpy<2" and pushed out a new version of splinecalib (0.0.8). From my local testing this seems to work (and avoids pinning a version). Interestingly, if you have numpy 2.0.0 installed already, this doesn't cause any problem - it seems to let you continue with that version.

If anyone is strong at the nitty-gritty on packaging, I'd love some help advice on how to clean this stuff up...

In the meantime, please let me know if this is now working reasonably well with version 0.0.8 (@SimpleCoder35 @Jennie-Myhren)

Jennie-Myhren commented 2 weeks ago

@blucena - Thanks for the fix! splinecalib v0.0.8 installs without issue when I run poetry install inside my docker container.

My latest projects only use pyproject.toml, but I found this guide helpful when I was playing around with different potential fixes for the numpy dependency issue: https://packaging.python.org/en/latest/guides/modernize-setup-py-project/.

dcturner45 commented 2 weeks ago

@blucena I am able to build successfully with the 0.0.8 release. Thanks so much for the quick fix! One last small issue - if I don't explicitly specify numpy<2.0.0 in my own package's requirements, I get the following fatal runtime error when trying to run my package:

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.0 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

I'm able to resolve this by specifying numpy<2.0.0 in my package's requirements, which is easy to do, but you may want to also specify numpy<2.0.0 in splinecalib's setup.py's install_requires to ensure the right numpy version is present at runtime. I'm not familiar with pyproject, but it seems in this case the dependencies listed in pyproject.toml are "build-time", while setup.py's dependencies are "runtime". They are apparently resolved independently of each other, so it looks like version-gating needs to be explicitly specified in both.

blucena commented 2 weeks ago

@Jennie-Myhren Thanks for confirming it works in your setup. @dcturner45 Also thanks for confirming. I did notice in my environment that it allowed me to work with numpy 2.0 despite the restriction in pyproject.toml and that it was likely because of the install_requires piece in setup.py. i'm going to experiment a little bit, but if it seems the right choice I'll add the "numpy<2" to setup.py also and push out a new version.

blucena commented 2 weeks ago

@dcturner45 @Jennie-Myhren @SimpleCoder35

Added "numpy<2" to the setup.py and pushed out version 0.0.9

Thanks for all your help and let me know if you have any other feedback / suggestions.