Pip install build fails, can't find mpc.h

[smprather]

I installed libmpc and libmpc-devel in my own space and wrote a .pc file, and set PKG_CONFIG_PATH accordingly.

(python3.12) $ pkg-config --cflags --libs libmpc
-I/proj/cot_globals/centos8/usr/include -L/proj/cot_globals/centos8/usr/lib64 -lmpc

(python3.12) $ ls /proj/cot_globals/centos8/usr/include
total 16
-rw-r--r-- 1 epramyl eusers 12898 Jan 11  2018 mpc.h

But the build can't find it. I did some Googling on how pip/setuptools interacts with external lib requirements, but I couldn't find much.

(python3.12) $ pip install gmpy2
Looking in indexes: https://arm.sero.gic.ericsson.se/artifactory/api/pypi/pypi-remote/simple/
Collecting gmpy2
  Using cached https://arm.sero.gic.ericsson.se/artifactory/api/pypi/pypi-remote/packages/packages/d9/2e/2848cb5ab5240cb34b967602990450d0fd715f013806929b2f82821cef7f/gmpy2-2.1.5.tar.gz (261 kB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: gmpy2
  Building wheel for gmpy2 (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [30 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-312
      creating build/lib.linux-x86_64-cpython-312/gmpy2
      copying gmpy2/__init__.py -> build/lib.linux-x86_64-cpython-312/gmpy2
      running egg_info
      writing gmpy2.egg-info/PKG-INFO
      writing dependency_links to gmpy2.egg-info/dependency_links.txt
      writing top-level names to gmpy2.egg-info/top_level.txt
      reading manifest file 'gmpy2.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      adding license file 'COPYING'
      adding license file 'COPYING.LESSER'
      writing manifest file 'gmpy2.egg-info/SOURCES.txt'
      copying gmpy2/__init__.pxd -> build/lib.linux-x86_64-cpython-312/gmpy2
      copying gmpy2/gmpy2.h -> build/lib.linux-x86_64-cpython-312/gmpy2
      copying gmpy2/gmpy2.pxd -> build/lib.linux-x86_64-cpython-312/gmpy2
      running build_ext
      building 'gmpy2.gmpy2' extension
      creating build/temp.linux-x86_64-cpython-312
      creating build/temp.linux-x86_64-cpython-312/src
      gcc -pthread -fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall -fPIC -I./src -I/proj/cot_globals/envs/python3.12/include -I/proj/cot_globals/tools/freeware/python/3.12.0/include/python3.12 -c src/gmpy2.c -o build/temp.linux-x86_64-cpython-312/src/gmpy2.o -DSHARED=1
      In file included from src/gmpy2.c:517:
      src/gmpy2.h:83:10: fatal error: mpc.h: No such file or directory
         83 | #include <mpc.h>
            |          ^~~~~~~
      compilation terminated.
      error: command '/app/vbuild/RHEL8-x86_64/gcc/12.2.0/bin/gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for gmpy2
  Running setup.py clean for gmpy2
Failed to build gmpy2
ERROR: Could not build wheels for gmpy2, which is required to install pyproject.toml-based projects

I poked around in another project's setup.py and found this.

# How the build process finds the system libs:
#
# 1. if BORG_{LIBXXX,OPENSSL}_PREFIX is set, it will use headers and libs from there.
# 2. if not and pkg-config can locate the lib, the lib located by
#    pkg-config will be used. We use the pkg-config tool via the pkgconfig
#    python package, which must be installed before invoking setup.py.
#    if pkgconfig is not installed, this step is skipped.
# 3. otherwise raise a fatal error.

    try:
        import pkgconfig as pc
    except ImportError:
        print("Warning: can not import pkgconfig python package.")
        pc = None

    def lib_ext_kwargs(pc, prefix_env_var, lib_name, lib_pkg_name, pc_version, lib_subdir="lib"):
        system_prefix = os.environ.get(prefix_env_var)
        if system_prefix:
            print(f"Detected and preferring {lib_pkg_name} [via {prefix_env_var}]")
            return dict(
                include_dirs=[os.path.join(system_prefix, "include")],
                library_dirs=[os.path.join(system_prefix, lib_subdir)],
                libraries=[lib_name],
            )

        if pc and pc.installed(lib_pkg_name, pc_version):
            print(f"Detected and preferring {lib_pkg_name} [via pkg-config]")
            return pc.parse(lib_pkg_name)
        raise Exception(
            f"Could not find {lib_name} lib/headers, please set {prefix_env_var} "
            f"or ensure {lib_pkg_name}.pc is in PKG_CONFIG_PATH."
        )

Thanks for the help!

[skirpichev]

Could you try

CFLAGS=-I<...> LDFLAGS=-L<...> pip install gmpy2

?

[skirpichev]

See also pypa/setuptools#3794.

I don't think that adding new build deps on pkgconfig will be a good idea for gmpy2. Rather we could document using compiler/linker flags to point on non-standard locations.

[smprather]

Can you just supply a wheel for my platform? I would much rather compilation not be attempted since it will just about always leads to "works ok on my machine" syndrome.

[skirpichev]

@smprather, unfortunately we can't guess your platform nor the gmpy2 version you are trying to install... Could you try latest alpha?

[smprather]

I pip install LOTS of pypi packages that have binary components but do not build during install. How are they able to guess my platform? For reference, it is Redhat EL 8.7 (Ootpa). I would think this is a fairly common one, especially in the enterprise world.

I am behind a Great Firewall, as in zero internet visibility (when in Linux; my Windows laptop is less restricted). We have a PyPI internal mirror on the Linux side. I cannot install arbitrary GitHub release/commits. Also, due to VM restrictions on my laptop, I cannot install Redhat 8.7. I can barely get WSL/Ubuntu working. I can run Docker from WSL, but I think that's beyond my pain point.

I could clone the alpha in WSL and transfer it over to the Redhat side. I may try that if I can find the time.

This is the only package I'm having trouble with so far, and it is only with Python 3.12. Hopefully you guys can catch up with everyone else that is shipping non-pure Python packages.

Thanks @skirpichev !

[skirpichev]

We have a PyPI internal mirror on the Linux side.

Is this a complete mirror or it has some restrictions (e.g. doesn't include alpha releases)?

If it has alpha versions - please try to do pip install gmpy2==2.2.0a1.

I cannot install arbitrary GitHub release/commits.

Latest gmpy2 alpha is available on the PyPI.

the only package I'm having trouble with so far, and it is only with Python 3.12

3.12 support is available only in 2.2.0a1.

There is #446 with a patch for 2.1.x, but I doubt 2.1.6 is planned to be released. @casevh ?

[skirpichev]

@smprather, now v2.2 is released. Does this issue is valid for you?

[smprather]

Works! Thanks @skirpichev !

(3.12.2) $ pip install gmpy2
DEPRECATION: Loading egg at /******************/python_venv/3.12.2/lib/python3.12/site-packages/gdspy-1.6.13-py3.12-linux-x86_64.egg is deprecated. pip 24.3 will enforce this behaviour change. A possible replacement is to use pip for package installation. Discussion can be found at https://github.com/pypa/pip/issues/12330
Looking in indexes: https://***************/artifactory/api/pypi/pypi-remote/simple/
Collecting gmpy2
  Downloading https://*************/artifactory/api/pypi/pypi-remote/packages/packages/96/8d/d017599d6db8e9b96d6e84ea5102c33525cb71c82876b1813a2ece5d94ec/gmpy2-2.2.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 57.6 MB/s eta 0:00:00
Installing collected packages: gmpy2
Successfully installed gmpy2-2.2.1
(3.12.2) $

[skirpichev]

Then, maybe this should be closed?