MAINT: NumPy complex shim needed?

[tylerjereddy]

Following https://github.com/numpy/numpy/pull/24376, I'm having trouble building SciPy alongside NumPy main, with errors in Pythran that may relate to NumPy 2.0-geared API changes? Most likely complex-related changes?

[778/1588] Compiling C++ object scipy/stats/_stats_pythran.cpython-311-x86_64-linux-gnu.so.p/meson-generated_..__stats_pythran.cpp.o
FAILED: scipy/stats/_stats_pythran.cpython-311-x86_64-linux-gnu.so.p/meson-generated_..__stats_pythran.cpp.o 
c++ -Iscipy/stats/_stats_pythran.cpython-311-x86_64-linux-gnu.so.p -Iscipy/stats -I../scipy/stats -I../../../python_venvs/py_311_scipy_dev/lib/python3.11/site-packages/pythran -I../../../python_venvs/py_311_scipy_dev/lib/python3.11/site-packages/numpy/core/include -I/home/treddy/github_projects/spack/opt/spack/linux-ubuntu22.04-skylake/gcc-11.3.0/python-3.11.2-4qncg2nqev5evxfmamde3e6rnb34b4ls/include/python3.11 -fvisibility=hidden -fvisibility-inlines-hidden -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -std=c++14 -O2 -g -fPIC -DENABLE_PYTHON_MODULE -D__PYTHRAN__=3 -DPYTHRAN_BLAS_NONE -Wno-cpp -Wno-deprecated-declarations -Wno-unused-but-set-variable -Wno-unused-function -Wno-unused-variable -Wno-int-in-bool-context -MD -MQ scipy/stats/_stats_pythran.cpython-311-x86_64-linux-gnu.so.p/meson-generated_..__stats_pythran.cpp.o -MF scipy/stats/_stats_pythran.cpython-311-x86_64-linux-gnu.so.p/meson-generated_..__stats_pythran.cpp.o.d -o scipy/stats/_stats_pythran.cpython-311-x86_64-linux-gnu.so.p/meson-generated_..__stats_pythran.cpp.o -c scipy/stats/_stats_pythran.cpp
In file included from ../../../python_venvs/py_311_scipy_dev/lib/python3.11/site-packages/pythran/pythonic/numpy/complex64.hpp:6,
                 from ../../../python_venvs/py_311_scipy_dev/lib/python3.11/site-packages/pythran/pythonic/types/ndarray.hpp:25,
                 from ../../../python_venvs/py_311_scipy_dev/lib/python3.11/site-packages/pythran/pythonic/types/tuple.hpp:8,
                 from ../../../python_venvs/py_311_scipy_dev/lib/python3.11/site-packages/pythran/pythonic/builtins/bool_.hpp:6,
                 from ../../../python_venvs/py_311_scipy_dev/lib/python3.11/site-packages/pythran/pythonic/types/NoneType.hpp:6,
                 from ../../../python_venvs/py_311_scipy_dev/lib/python3.11/site-packages/pythran/pythonic/types/slice.hpp:5,
                 from ../../../python_venvs/py_311_scipy_dev/lib/python3.11/site-packages/pythran/pythonic/core.hpp:46,
                 from scipy/stats/_stats_pythran.cpp:1:
../../../python_venvs/py_311_scipy_dev/lib/python3.11/site-packages/pythran/pythonic/types/complex.hpp: In static member function ‘static std::complex<_Tp> {anonymous}::pythonic::from_python<std::complex<_Tp> >::convert(PyObject*) [with T = long double; PyObject = _object]’:
../../../python_venvs/py_311_scipy_dev/lib/python3.11/site-packages/pythran/pythonic/types/complex.hpp:231:15: error: request for member ‘real’ in ‘val’, which is of non-class type ‘__complex__ long double’
  231 |   return {val.real, val.imag};
      |               ^~~~
../../../python_venvs/py_311_scipy_dev/lib/python3.11/site-packages/pythran/pythonic/types/complex.hpp:231:25: error: request for member ‘imag’ in ‘val’, which is of non-class type ‘__complex__ long double’
  231 |   return {val.real, val.imag};
      |                         ^~~~
../../../python_venvs/py_311_scipy_dev/lib/python3.11/site-packages/pythran/pythonic/types/complex.hpp:231:29: error: could not convert ‘{<expression error>, <expression error>}’ from ‘<brace-enclosed initializer list>’ to ‘std::complex<long double>’
  231 |   return {val.real, val.imag};
      |                             ^
      |                             |
      |                             <brace-enclosed initializer list>
../../../python_venvs/py_311_scipy_dev/lib/python3.11/site-packages/pythran/pythonic/types/complex.hpp: In static member function ‘static std::complex<_Tp> {anonymous}::pythonic::from_python<std::complex<_Tp> >::convert(PyObject*) [with T = float; PyObject = _object]’:
../../../python_venvs/py_311_scipy_dev/lib/python3.11/site-packages/pythran/pythonic/types/complex.hpp:246:15: error: request for member ‘real’ in ‘val’, which is of non-class type ‘__complex__ float’
  246 |   return {val.real, val.imag};
      |               ^~~~
../../../python_venvs/py_311_scipy_dev/lib/python3.11/site-packages/pythran/pythonic/types/complex.hpp:246:25: error: request for member ‘imag’ in ‘val’, which is of non-class type ‘__complex__ float’
  246 |   return {val.real, val.imag};
      |                         ^~~~
../../../python_venvs/py_311_scipy_dev/lib/python3.11/site-packages/pythran/pythonic/types/complex.hpp:246:29: error: could not convert ‘{<expression error>, <expression error>}’ from ‘<brace-enclosed initializer list>’ to ‘std::complex<float>’
  246 |   return {val.real, val.imag};
      |                             ^
      |                             |
      |                             <brace-enclosed initializer list>

cc @rgommers @ngoldbaum @mtsokol

[ngoldbaum]

pythran/pythonic/types/complex.hpp:231:15: error: request for member ‘real’ in ‘val’, which is of non-class type ‘__complex__ long double’

ping @lysnikolaou

Could this be related to changes to numpy's wrappers for the C99 complex types?

See https://github.com/numpy/numpy/pull/24085

[rgommers]

This was already fixed in gh-2128. If you use a numpy nightly, you should temporarily install Pythran from latest master, just like the SciPy pre-release job does.

There's nothing else to do here I think, aside from waiting for a new Pythran release and bumping the minimum version needed in SciPy.

[tylerjereddy]

To be fair, the pythran master branch testsuite failed for me locally with NumPy main so I just gave up on it--I guess I should have tried using it anyway.

111 failed, 3402 passed, 56 skipped, 65 warnings in 2603.89s (0:43:23)

Many of the test failures proceed through distutils.errors.CompileError, but I think the problem is that pythran can't dig around for my OpenBLAS without using a ~/.pythranrc.