Pip installation broken

[MPenaR]

When performing pip install numba-scipy on a clean enviroment, the installation fails with a very long error ending in:

.
.
.
      INFO: CCompilerOpt.cache_flush[863] : write cache to path -> /tmp/pip-install-g8a5sb7r/scipy_146ee8793f624beeb0b89a9ff607c49a/build/temp.linux-x86_64-3.11/ccompiler_opt_cache_clib.py
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for scipy
Failed to build scipy
ERROR: Could not build wheels for scipy, which is required to install pyproject.toml-based projects

python version is 3.11.6

[esc]

@MPenaR thank you for your report. Can you please append the complete log to this report, thank you.

[MPenaR]

Sure, I've redirected separately the stderr and the stdout as the stderr is so large.

stdout:

Collecting numba-scipy
  Using cached numba_scipy-0.3.1-py3-none-any.whl (7.4 kB)
Collecting scipy<=1.7.3,>=0.16 (from numba-scipy)
  Using cached scipy-1.6.1.tar.gz (27.3 MB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting numba>=0.45 (from numba-scipy)
  Using cached numba-0.58.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.metadata (2.7 kB)
Collecting llvmlite<0.42,>=0.41.0dev0 (from numba>=0.45->numba-scipy)
  Using cached llvmlite-0.41.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.8 kB)
Collecting numpy<1.27,>=1.22 (from numba>=0.45->numba-scipy)
  Using cached numpy-1.26.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)
Using cached numba-0.58.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (3.6 MB)
Using cached llvmlite-0.41.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (43.6 MB)
Using cached numpy-1.26.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)
Building wheels for collected packages: scipy
  Building wheel for scipy (pyproject.toml): started
  Building wheel for scipy (pyproject.toml): finished with status 'error'
Failed to build scipy

the stderr are more than 10K lines, is there any way I can share that with you? or maybe some option to pip install to make it less verbose?

[esc]

@MPenaR not sure about the stderr log, but, I think you can attach a text file to an issues.

Also, it looks like it is failing to install scipy in the first place?

[MPenaR]

Here are the attached logs. output.log error.log

by the way, scipy installs without any problem:

pip install scipy
Collecting scipy
  Using cached scipy-1.11.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (60 kB)
Collecting numpy<1.28.0,>=1.21.6 (from scipy)
  Using cached numpy-1.26.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (61 kB)
Using cached scipy-1.11.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (36.4 MB)
Using cached numpy-1.26.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (18.2 MB)
Installing collected packages: numpy, scipy
Successfully installed numpy-1.26.1 scipy-1.11.3

[esc]

The error is:

      scipy/cluster/_vq.c:196:12: fatal error: longintrepr.h: No existe el fichero o el directorio
        196 |   #include "longintrepr.h"
            |            ^~~~~~~~~~~~~~~
      compilation terminated.
      error: Command "x86_64-linux-gnu-gcc -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection -g -fwrapv -O2 -fPIC -I/tmp/pip-build-env-vu1mt6b9/overlay/lib/python3.11/site-packages/numpy/core/include -I/tmp/pip-build-env-vu1mt6b9/overlay/lib/python3.11/site-packages/numpy/core/include -Ibuild/src.linux-x86_64-3.11/numpy/distutils/include -I/home/manuel/.local/share/virtualenvs/numba_scipy-ZLtI8Kyr/include -I/usr/include/python3.11 -c scipy/cluster/_vq.c -o build/temp.linux-x86_64-3.11/scipy/cluster/_vq.o -MMD -MF build/temp.linux-x86_64-3.11/scipy/cluster/_vq.o.d -msse -msse2 -msse3" failed with exit status 1

Which leads me to two questions:

a) why is pip trying to install scipy by compiling it rather than installing the wheel? b) why is the compilation failing?

Also note, that when you pip install scipy you get 1.11.3 but when you pip install numba-scipy it seems like it tries to install 1.6.1. My guess is that the lack of a recent numba-scipy which updates the maximum scipy dependency is perhaps an issue here: https://github.com/numba/numba-scipy/issues/97.

As a workaround you can try to install numba-scipy from git directly. You should be able to google for instructions on how to do that.

[MPenaR]

thanks, that worked:

Successfully installed llvmlite-0.41.1 numba-0.58.1 numba-scipy-0.4.0.dev0+56.g1e2f244 numpy-1.26.1 scipy-1.10.1

[esc]

thanks, that worked:

Successfully installed llvmlite-0.41.1 numba-0.58.1 numba-scipy-0.4.0.dev0+56.g1e2f244 numpy-1.26.1 scipy-1.10.1

Awesome, thank you for trying and thank you for reporting back. Good to have some evidence that this is related to dependency management.

[guywilsonjr]

Fyi @esc I've seen this issue before. I can't remember the exact problem, but Fails to build on Python 3.11 - longintrepr.h: No such file or directory looks relevant. Particularly the note:

Okay, I was able to fix the problem. It turned out that I depended on some package which depended on an old version of cython

thank you this solve my problem, i just install it with pip3 install cython

[esc]

Fyi @esc I've seen this issue before. I can't remember the exact problem, but Fails to build on Python 3.11 - longintrepr.h: No such file or directory looks relevant. Particularly the note:

Okay, I was able to fix the problem. It turned out that I depended on some package which depended on an old version of cython

thank you this solve my problem, i just install it with pip3 install cython

@guywilsonjr thank you for adding this comment, may indeed be relevant. Still hoping these issues will go away with the next release.

[esc]

This should now be resolved as 0.4.0 is available on PyPi -- please check and report back, if it works for you.