Open chnce opened 7 months ago
How are you installing the various packages? All with conda? Or other ways?
Thanks for the quick response. I detailed the steps in the original post above.
Given that you are reporting a clash with numpy and openmp binaries from conda forge, then I think you should open the issue at one or both of these locations:
https://github.com/conda-forge/openmp-feedstock
https://github.com/conda-forge/numpy-feedstock
I don't see a slycot specific issue (yet).
I agree that it's not a true issue of slycot. But maybe slycot could move to a more modern version of libflang. I have little hope, that anyone at openmp is going to fix a 6 year old version.
I could frankenstein together a working environment by ignoring the dependencies and installing:
Ah, ok I understand now. slycot forces an old libflang and thus forces you to install an old openmp, which has an incompatibility with numpy. So yes, it would be helpful to move slycot to a newer libflang.
It doesn't look like this is something we explicitly set:
https://github.com/conda-forge/slycot-feedstock/blob/main/recipe/meta.yaml
It may mean that conda forge requires libfang<6 from the {{ compiler('fortran') }}
call. If so, then this would be a bug to report to the numpy and openmp feedstocks.
It looks locked to libflang 5 on windows across all of conda-forge: https://github.com/conda-forge/conda-forge-pinning-feedstock/blob/main/recipe/conda_build_config.yaml#L30
Yes we try to follow the conda-forge pinning for flang. See also https://github.com/conda-forge/slycot-feedstock/pull/65 and https://github.com/conda-forge/blas-feedstock/issues/106
Thank you for your insights. I filed the issue with numpy and openmp feedstock. I'm genuinely surprised that no one else seems to have the same issue. openmp 5 was downloaded half a million times from conda-forge.
There is a similar problem under Debian. Using the installed python3-numpy package (numpy version 1.24.2 in trixie)
pip install slycot
doesn't work correctly (building stops). The only way to install slycot ist to force the removing of python3-numpy
sudo dpkg -P --force-all python3-numpy)
and install numpy using pip (in this case the installed version is numpy-1.26.4).After this step
pip install spycot
works.
Could you post a build log with the error message, please?
Tested on a docker after clonng Slycot in git, but it correclty works after updating numpy. In the pyproject.toml the version of rerquired numpy is numpy >= 1.23.1 an in Debian we have 1.24.2.
root@debian:/psc/Slycot# pip install --break-system-packages . Processing /psc/Slycot Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Requirement already satisfied: numpy>=1.23.1 in /usr/lib/python3/dist-packages (from slycot==0.5.5.dev60+gfea37c7) (1.24.2) Building wheels for collected packages: slycot Building wheel for slycot (pyproject.toml) ... error error: subprocess-exited-with-error
�� Building wheel for slycot (pyproject.toml) did not run successfully. ��� exit code: 1 ������> [166 lines of output]
--------------------------------------------------------------------------------
-- Trying 'Ninja' generator
--------------------------------
---------------------------
----------------------
-----------------
------------
-------
--
CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
Compatibility with CMake < 3.5 will be removed from a future version of
CMake.
Update the VERSION argument <min> value or use a ...<max> suffix to tell
CMake that the project does not need compatibility with older versions.
Not searching for unused variables given on the command line.
CMake Error: CMake was unable to find a build program corresponding to "Ninja". CMAKE_MAKE_PROGRAM is not set. You probably need to select a different build tool.
-- Configuring incomplete, errors occurred!
--
-------
------------
-----------------
----------------------
---------------------------
--------------------------------
-- Trying 'Ninja' generator - failure
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
-- Trying 'Unix Makefiles' generator
--------------------------------
---------------------------
----------------------
-----------------
------------
-------
--
CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
Compatibility with CMake < 3.5 will be removed from a future version of
CMake.
Update the VERSION argument <min> value or use a ...<max> suffix to tell
CMake that the project does not need compatibility with older versions.
Not searching for unused variables given on the command line.
-- The C compiler identification is GNU 13.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- The Fortran compiler identification is GNU 13.2.0
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Check for working Fortran compiler: /usr/bin/gfortran - skipped
-- Configuring done (0.2s)
-- Generating done (0.0s)
-- Build files have been written to: /psc/Slycot/_cmake_test_compile/build
--
-------
------------
-----------------
----------------------
---------------------------
--------------------------------
-- Trying 'Unix Makefiles' generator - success
--------------------------------------------------------------------------------
Configuring Project
Working directory:
/psc/Slycot/_skbuild/linux-x86_64-3.11/cmake-build
Command:
/tmp/pip-build-env-vj8gsmqn/overlay/local/lib/python3.11/dist-packages/cmake/data/bin/cmake /psc/Slycot -G 'Unix Makefiles' --no-warn-unused-cli -DCMAKE_INSTALL_PREFIX:PATH=/psc/Slycot/_skbuild/linux-x86_64-3.11/cmake-install -DPYTHON_VERSION_STRING:STRING=3.11.8 -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/tmp/pip-build-env-vj8gsmqn/overlay/local/lib/python3.11/dist-packages/skbuild/resources/cmake -DPYTHON_EXECUTABLE:PATH=/usr/bin/python3 -DPYTHON_INCLUDE_DIR:PATH=/usr/include/python3.11 -DPYTHON_LIBRARY:PATH=/usr/lib/x86_64-linux-gnu/libpython3.11.so -DPython_EXECUTABLE:PATH=/usr/bin/python3 -DPython_ROOT_DIR:PATH=/usr -DPython_FIND_REGISTRY:STRING=NEVER -DPython_INCLUDE_DIR:PATH=/usr/include/python3.11 -DPython_NumPy_INCLUDE_DIRS:PATH=/tmp/pip-build-env-vj8gsmqn/overlay/local/lib/python3.11/dist-packages/numpy/core/include -DPython3_EXECUTABLE:PATH=/usr/bin/python3 -DPython3_ROOT_DIR:PATH=/usr -DPython3_FIND_REGISTRY:STRING=NEVER -DPython3_INCLUDE_DIR:PATH=/usr/include/python3.11 -DPython3_NumPy_INCLUDE_DIRS:PATH=/tmp/pip-build-env-vj8gsmqn/overlay/local/lib/python3.11/dist-packages/numpy/core/include -DCMAKE_BUILD_TYPE:STRING=Release
Not searching for unused variables given on the command line.
-- The C compiler identification is GNU 13.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- The Fortran compiler identification is GNU 13.2.0
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Check for working Fortran compiler: /usr/bin/gfortran - skipped
-- Found Python: /usr/bin/python3 (found version "3.11.8") found components: Interpreter Development NumPy Development.Module Development.Embed
CMake Warning (dev) at /tmp/pip-build-env-vj8gsmqn/overlay/local/lib/python3.11/dist-packages/skbuild/resources/cmake/FindPythonExtensions.cmake:245 (find_package):
Policy CMP0148 is not set: The FindPythonInterp and FindPythonLibs modules
are removed. Run "cmake --help-policy CMP0148" for policy details. Use
the cmake_policy command to set the policy and suppress this warning.
Call Stack (most recent call first):
CMakeLists.txt:13 (find_package)
This warning is for project developers. Use -Wno-dev to suppress it.
-- Found PythonInterp: /usr/bin/python3 (found version "3.11.8")
CMake Warning (dev) at /tmp/pip-build-env-vj8gsmqn/overlay/local/lib/python3.11/dist-packages/skbuild/resources/cmake/FindPythonExtensions.cmake:252 (find_package):
Policy CMP0148 is not set: The FindPythonInterp and FindPythonLibs modules
are removed. Run "cmake --help-policy CMP0148" for policy details. Use
the cmake_policy command to set the policy and suppress this warning.
Call Stack (most recent call first):
CMakeLists.txt:13 (find_package)
This warning is for project developers. Use -Wno-dev to suppress it.
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython3.11.so (found version "3.11.8")
-- Found NumPy: /tmp/pip-build-env-vj8gsmqn/overlay/local/lib/python3.11/dist-packages/numpy/core/include (found version "1.26.4")
Traceback (most recent call last):
File "/usr/bin/f2py3", line 33, in <module>
sys.exit(load_entry_point('numpy==1.24.2', 'console_scripts', 'f2py3.11')())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/bin/f2py3", line 25, in importlib_load_entry_point
return next(matches).load()
^^^^^^^^^^^^^
StopIteration
-- Found F2PY: /usr/bin/f2py3 (found version "")
-- Looking for Fortran sgemm
-- Looking for Fortran sgemm - not found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Looking for Fortran sgemm
-- Looking for Fortran sgemm - found
-- Found BLAS: /usr/lib/x86_64-linux-gnu/libopenblas.so
-- Looking for Fortran cheev
-- Looking for Fortran cheev - found
-- Found LAPACK: /usr/lib/x86_64-linux-gnu/libopenblas.so;-lm;-ldl
-- Python headers included from: /usr/include/python3.11
-- NumPy headers included from: /tmp/pip-build-env-vj8gsmqn/overlay/local/lib/python3.11/dist-packages/numpy/core/include
-- F2PY headers included from: /tmp/pip-build-env-vj8gsmqn/overlay/local/lib/python3.11/dist-packages/numpy/f2py/src;/tmp/pip-build-env-vj8gsmqn/overlay/local/lib/python3.11/dist-packages/numpy/core/include
-- LAPACK: /usr/lib/x86_64-linux-gnu/libopenblas.so;-lm;-ldl
-- BLAS: /usr/lib/x86_64-linux-gnu/libopenblas.so
CMake Error at CMakeLists.txt:29 (message):
Python include directory has a numpy sub-directory,
/usr/include/python3.11/numpy,
which is different from Numpy include directory
/tmp/pip-build-env-vj8gsmqn/overlay/local/lib/python3.11/dist-packages/numpy/core/include.
You're probably building in a virtual environment, in which case
uninstall numpy from the base environment and try again.
-- Configuring incomplete, errors occurred!
Traceback (most recent call last):
File "/tmp/pip-build-env-vj8gsmqn/overlay/local/lib/python3.11/dist-packages/skbuild/setuptools_wrap.py", line 666, in setup
env = cmkr.configure(
^^^^^^^^^^^^^^^
File "/tmp/pip-build-env-vj8gsmqn/overlay/local/lib/python3.11/dist-packages/skbuild/cmaker.py", line 357, in configure
raise SKBuildError(msg)
An error occurred while configuring with CMake.
Command:
/tmp/pip-build-env-vj8gsmqn/overlay/local/lib/python3.11/dist-packages/cmake/data/bin/cmake /psc/Slycot -G 'Unix Makefiles' --no-warn-unused-cli -DCMAKE_INSTALL_PREFIX:PATH=/psc/Slycot/_skbuild/linux-x86_64-3.11/cmake-install -DPYTHON_VERSION_STRING:STRING=3.11.8 -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/tmp/pip-build-env-vj8gsmqn/overlay/local/lib/python3.11/dist-packages/skbuild/resources/cmake -DPYTHON_EXECUTABLE:PATH=/usr/bin/python3 -DPYTHON_INCLUDE_DIR:PATH=/usr/include/python3.11 -DPYTHON_LIBRARY:PATH=/usr/lib/x86_64-linux-gnu/libpython3.11.so -DPython_EXECUTABLE:PATH=/usr/bin/python3 -DPython_ROOT_DIR:PATH=/usr -DPython_FIND_REGISTRY:STRING=NEVER -DPython_INCLUDE_DIR:PATH=/usr/include/python3.11 -DPython_NumPy_INCLUDE_DIRS:PATH=/tmp/pip-build-env-vj8gsmqn/overlay/local/lib/python3.11/dist-packages/numpy/core/include -DPython3_EXECUTABLE:PATH=/usr/bin/python3 -DPython3_ROOT_DIR:PATH=/usr -DPython3_FIND_REGISTRY:STRING=NEVER -DPython3_INCLUDE_DIR:PATH=/usr/include/python3.11 -DPython3_NumPy_INCLUDE_DIRS:PATH=/tmp/pip-build-env-vj8gsmqn/overlay/local/lib/python3.11/dist-packages/numpy/core/include -DCMAKE_BUILD_TYPE:STRING=Release
Source directory:
/psc/Slycot
Working directory:
/psc/Slycot/_skbuild/linux-x86_64-3.11/cmake-build
Please see CMake's output for more information.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for slycot Failed to build slycot ERROR: Could not build wheels for slycot, which is required to install pyproject.toml-based projects root@debian:/psc/Slycot#
After installing slycot in a conda environment, a kernel crash occurs when trying to solve a linear system with a singular matrix.
I narrowed the issue down to the following:
Openmp places its "libiomp5md.dll" in Library\bin. The version from numpy is either in Library\bin (channel main) or directly in bin (conda-forge).
Actual Behavior
Using numpy.linalg.solve on singular matrix with dimension over 149 crashes the Kernel.
Expected Behavior
Expected a "LinAlgError: Singular matrix" as it is shown with matrices up to the dimension of 149.
Steps to Reproduce
mamba create -n slycot_crash python=3.11 numpy ipython -c conda-forge
mamba activate slycot_crash
ipython
exit
mamba install openmp=5 -c conda-forge