search

pdm-project / pdm

A modern Python package and dependency manager supporting the latest PEP standards
https://pdm-project.org
MIT License
7.03k stars 352 forks source link

pdm update failing on shapely #2857

Closed pythonmobile closed 1 month ago

pythonmobile commented 1 month ago

Make sure you run commands with -v flag before pasting the output.

Steps to reproduce

On Python 3.12.3 (on ubuntu 22.04), pdm update is failing on the following pyproject.toml:

[project]
name = "myproj"
version = "1.1.1"
description = "Mongo"
 dependencies = [
     "shapely",
 ]
requires-python = ">=3.8"

But python3 -m pip install shapely # works flawlessly.

Actual behavior

# pdm update -v
STATUS: Resolving dependencies
pdm.termui: ======== Start resolving requirements ========
pdm.termui:   appdirs>=1.4.4
pdm.termui:   mongoengine>=0.28.2
pdm.termui:   pymongo>=4.7.0
pdm.termui:   pyyaml>=6.0.1
pdm.termui:   shapely
pdm.termui:   python>=3.8
pdm.termui:   Adding requirement appdirs>=1.4.4
pdm.termui:   Adding requirement mongoengine>=0.28.2
pdm.termui:   Adding requirement pymongo>=4.7.0
pdm.termui:   Adding requirement pyyaml>=6.0.1
pdm.termui:   Adding requirement shapely
pdm.termui:   Adding requirement python>=3.8
pdm.termui: ======== Starting round 0 ========
STATUS: Resolving: new pin python>=3.8
pdm.termui: Pinning: python None
pdm.termui: ======== Ending round 0 ========
pdm.termui: ======== Starting round 1 ========
STATUS: Resolving: new pin appdirs 1.4.4
pdm.termui: Pinning: appdirs 1.4.4
pdm.termui: ======== Ending round 1 ========
pdm.termui: ======== Starting round 2 ========
pdm.termui:   Adding requirement pymongo<5.0,>=3.4(from mongoengine 0.28.2)
STATUS: Resolving: new pin mongoengine 0.28.2
pdm.termui: Pinning: mongoengine 0.28.2
pdm.termui: ======== Ending round 2 ========
pdm.termui: ======== Starting round 3 ========
pdm.termui:   Adding requirement dnspython<3.0.0,>=1.16.0(from pymongo 4.7.0)
STATUS: Resolving: new pin pymongo 4.7.0
pdm.termui: Pinning: pymongo 4.7.0
pdm.termui: ======== Ending round 3 ========
pdm.termui: ======== Starting round 4 ========
STATUS: Resolving: new pin pyyaml 6.0.1
pdm.termui: Pinning: pyyaml 6.0.1
pdm.termui: ======== Ending round 4 ========
pdm.termui: ======== Starting round 5 ========
pdm.termui:   Adding requirement numpy<3,>=1.14(from shapely 2.0.4)
STATUS: Resolving: new pin shapely 2.0.4
pdm.termui: Pinning: shapely 2.0.4
pdm.termui: ======== Ending round 5 ========
pdm.termui: ======== Starting round 6 ========
STATUS: Resolving: new pin dnspython 2.6.1
pdm.termui: Pinning: dnspython 2.6.1
pdm.termui: ======== Ending round 6 ========
pdm.termui: ======== Starting round 7 ========
STATUS: Resolving: new pin numpy 1.24.4
pdm.termui: Pinning: numpy 1.24.4
pdm.termui: ======== Ending round 7 ========
pdm.termui: ======== Starting round 8 ========
pdm.termui: ======== Resolution Result ========
pdm.termui: Stable pins:
pdm.termui:        python None
pdm.termui:       appdirs 1.4.4
pdm.termui:   mongoengine 0.28.2
pdm.termui:       pymongo 4.7.0
pdm.termui:        pyyaml 6.0.1
pdm.termui:       shapely 2.0.4
pdm.termui:     dnspython 2.6.1
pdm.termui:         numpy 1.24.4
STATUS: Fetching hashes for resolved packages...
pdm.termui: Fetching hashes for appdirs@1.4.4
pdm.termui: Fetching hashes for mongoengine@0.28.2
pdm.termui: Fetching hashes for pyyaml@6.0.1
pdm.termui: Fetching hashes for shapely@2.0.4
pdm.termui: Fetching hashes for pymongo@4.7.0
🔒 Lock successful
STATUS: Resolving packages from lockfile...
unearth.preparer: The file . is a local directory, use it directly
Synchronizing working set with resolved packages: 1 to add, 0 to update, 0 to remove

pdm.termui: Using cached response for https://files.pythonhosted.org/packages/a4/9b/027bec52c633f6556dba6b722d9a0befb40498b9ceddd29cbe67a45a127c/numpy-1.24.4.tar.gz
unearth.preparer: Downloading <Link https://files.pythonhosted.org/packages/a4/9b/027bec52c633f6556dba6b722d9a0befb40498b9ceddd29cbe67a45a127c/numpy-1.24.4.tar.gz (from https://pypi.org/simple/numpy/)> (10.9 MB)
pdm.termui: Preparing environment(Isolated mode) for PEP 517 build...
pdm.termui: Saving wheel to cache: <Link https://files.pythonhosted.org/packages/a4/9b/027bec52c633f6556dba6b722d9a0befb40498b9ceddd29cbe67a45a127c/numpy-1.24.4.tar.gz (from https://pypi.org/simple/numpy/)>
pdm.termui: Running PEP 517 backend to build a wheel for <Link https://files.pythonhosted.org/packages/a4/9b/027bec52c633f6556dba6b722d9a0befb40498b9ceddd29cbe67a45a127c/numpy-1.24.4.tar.gz (from https://pypi.org/simple/numpy/)>
pdm.termui: ======== Start resolving requirements ========
pdm.termui:   Cython<3.0,>=0.29.30
pdm.termui:   setuptools==59.2.0
pdm.termui:   wheel==0.37.0
pdm.termui:   python==3.12.3
pdm.termui:   Adding requirement Cython<3.0,>=0.29.30
pdm.termui:   Adding requirement setuptools==59.2.0
pdm.termui:   Adding requirement wheel==0.37.0
pdm.termui:   Adding requirement python==3.12.3
pdm.termui: ======== Starting round 0 ========
pdm.termui: Pinning: python None
pdm.termui: ======== Ending round 0 ========
pdm.termui: ======== Starting round 1 ========
pdm.termui: Pinning: setuptools 59.2.0
pdm.termui: ======== Ending round 1 ========
pdm.termui: ======== Starting round 2 ========
pdm.termui: Pinning: wheel 0.37.0
pdm.termui: ======== Ending round 2 ========
pdm.termui: ======== Starting round 3 ========
pdm.termui: Pinning: Cython 0.29.37
pdm.termui: ======== Ending round 3 ========
pdm.termui: ======== Starting round 4 ========
pdm.termui: ======== Resolution Result ========
pdm.termui: Stable pins:
pdm.termui:       python None
pdm.termui:   setuptools 59.2.0
pdm.termui:        wheel 0.37.0
pdm.termui:       cython 0.29.37
pdm.termui: Installing cython@0.29.37...
pdm.termui: Using cached response for https://files.pythonhosted.org/packages/c9/aa/99a0eac01136c0c75feb3210d107c49f93d49d5cb97f19e99318b9ecefdd/Cython-0.29.37-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl
unearth.preparer: Downloading <Link https://files.pythonhosted.org/packages/c9/aa/99a0eac01136c0c75feb3210d107c49f93d49d5cb97f19e99318b9ecefdd/Cython-0.29.37-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl (from https://pypi.org/simple/cython/)> (2.0 MB)
pdm.termui: Installing setuptools@59.2.0...
pdm.termui: Using cached response for https://files.pythonhosted.org/packages/18/ad/ec41343a49a0371ea40daf37b1ba2c11333cdd121cb378161635d14b9750/setuptools-59.2.0-py3-none-any.whl
unearth.preparer: Downloading <Link https://files.pythonhosted.org/packages/18/ad/ec41343a49a0371ea40daf37b1ba2c11333cdd121cb378161635d14b9750/setuptools-59.2.0-py3-none-any.whl (from https://pypi.org/simple/setuptools/)> (952 kB)
pdm.termui: Installing wheel@0.37.0...
pdm.termui: Using cached response for https://files.pythonhosted.org/packages/04/80/cad93b40262f5d09f6de82adbee452fd43cdff60830b56a74c5930f7e277/wheel-0.37.0-py2.py3-none-any.whl
unearth.preparer: Downloading <Link https://files.pythonhosted.org/packages/04/80/cad93b40262f5d09f6de82adbee452fd43cdff60830b56a74c5930f7e277/wheel-0.37.0-py2.py3-none-any.whl (from https://pypi.org/simple/wheel/)> (35 kB)
pdm.termui: Synchronization complete.
  ✖ Install numpy 1.24.4 failed
pdm.termui: Error occurs:
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pdm/installers/synchronizers.py", line 282, in install_candidate
    self.manager.install(can)
  File "/usr/local/lib/python3.12/site-packages/pdm/installers/manager.py", line 33, in install
    prepared.build(),
    ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pdm/models/candidates.py", line 416, in build
    self._cached = Path(builder.build(build_dir, metadata_directory=self._metadata_dir))
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pdm/builders/wheel.py", line 20, in build
    requires = self._hook.get_requires_for_build_wheel(self.config_settings)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pyproject_hooks/_impl.py", line 166, in get_requires_for_build_wheel
    return self._call_hook('get_requires_for_build_wheel', {
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pyproject_hooks/_impl.py", line 321, in _call_hook
    raise BackendUnavailable(data.get('traceback', ''))
pyproject_hooks._impl.BackendUnavailable: Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/pyproject_hooks/_in_process/_in_process.py", line 77, in _build_backend
    obj = import_module(mod_path)
          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/tmp/pdm-build-env-gic98je7-shared/lib/python3.12/site-packages/setuptools/__init__.py", line 10, in <module>
    import distutils.core
ModuleNotFoundError: No module named 'distutils'

Retry failed jobs
pdm.termui: Preparing environment(Isolated mode) for PEP 517 build...
pdm.termui: Saving wheel to cache: <Link https://files.pythonhosted.org/packages/a4/9b/027bec52c633f6556dba6b722d9a0befb40498b9ceddd29cbe67a45a127c/numpy-1.24.4.tar.gz (from https://pypi.org/simple/numpy/)>
pdm.termui: Running PEP 517 backend to build a wheel for <Link https://files.pythonhosted.org/packages/a4/9b/027bec52c633f6556dba6b722d9a0befb40498b9ceddd29cbe67a45a127c/numpy-1.24.4.tar.gz (from https://pypi.org/simple/numpy/)>
  ✖ Install numpy 1.24.4 failed
pdm.termui: Error occurs:
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pdm/installers/synchronizers.py", line 282, in install_candidate
    self.manager.install(can)
  File "/usr/local/lib/python3.12/site-packages/pdm/installers/manager.py", line 33, in install
    prepared.build(),
    ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pdm/models/candidates.py", line 416, in build
    self._cached = Path(builder.build(build_dir, metadata_directory=self._metadata_dir))
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pdm/builders/wheel.py", line 20, in build
    requires = self._hook.get_requires_for_build_wheel(self.config_settings)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pyproject_hooks/_impl.py", line 166, in get_requires_for_build_wheel
    return self._call_hook('get_requires_for_build_wheel', {
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pyproject_hooks/_impl.py", line 321, in _call_hook
    raise BackendUnavailable(data.get('traceback', ''))
pyproject_hooks._impl.BackendUnavailable: Traceback (most recent call last):
  File "/usr/local/lib/python3.12/site-packages/pyproject_hooks/_in_process/_in_process.py", line 77, in _build_backend
    obj = import_module(mod_path)
          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1310, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/tmp/pdm-build-env-gic98je7-shared/lib/python3.12/site-packages/setuptools/__init__.py", line 10, in <module>
    import distutils.core
ModuleNotFoundError: No module named 'distutils'

Traceback (most recent call last):
  File "/usr/local/bin/pdm", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pdm/core.py", line 358, in main
    return core.main(args or sys.argv[1:])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pdm/core.py", line 276, in main
    raise cast(Exception, err).with_traceback(traceback) from None
  File "/usr/local/lib/python3.12/site-packages/pdm/core.py", line 271, in main
    self.handle(project, options)
  File "/usr/local/lib/python3.12/site-packages/pdm/core.py", line 207, in handle
    command.handle(project, options)
  File "/usr/local/lib/python3.12/site-packages/pdm/cli/commands/update.py", line 73, in handle
    self.do_update(
  File "/usr/local/lib/python3.12/site-packages/pdm/cli/commands/update.py", line 193, in do_update
    do_sync(
  File "/usr/local/lib/python3.12/site-packages/pdm/cli/actions.py", line 238, in do_sync
    synchronizer.synchronize()
  File "/usr/local/lib/python3.12/site-packages/pdm/installers/synchronizers.py", line 464, in synchronize
    raise InstallationError("Some package operations are not complete yet")
pdm.exceptions.InstallationError: Some package operations are not complete yet

Expected behavior

I am hoping it can resolve the dependency issues with the update.

Environment Information

# Paste the output of `pdm info && pdm info --env` below:
PDM version:
  2.15.1
Python Interpreter:
  /tmp/.venv/bin/python (3.12)
Project Root:
  /tmp
Local Packages:

{
  "implementation_name": "cpython",
  "implementation_version": "3.12.3",
  "os_name": "posix",
  "platform_machine": "x86_64",
  "platform_release": "6.5.0-28-generic",
  "platform_system": "Linux",
  "platform_version": "#29~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Apr  4 14:39:20 UTC 2",
  "python_full_version": "3.12.3",
  "platform_python_implementation": "CPython",
  "python_version": "3.12",
  "sys_platform": "linux"
}
frostming commented 1 month ago

On python 3.12 you can't run this successfully due to the outdated build tools

pip install numpy==1.24.4

It's pinned to this outdated version because you specify a wide requires-python that allows down to 3.8, which prevents newer versions of numpy to be resolved. Read more about this at https://frostming.com/en/2024/pdm-lockfile/ and I don't want to explain more.