search

pdm-project / pdm

A modern Python package and dependency manager supporting the latest PEP standards
https://pdm-project.org
MIT License
7.93k stars 398 forks source link

pdm add to multi-target lockfile adds wrong marker to other packages #3142

Closed momartini closed 2 weeks ago

momartini commented 2 months ago

Make sure you run commands with -v flag before pasting the output.

Steps to reproduce

pyproject.toml:

[project]
name = "pdm"
version = "0.1.0"
description = "Default template for PDM package"
authors = [{ name = "name", email = "email" }]

requires-python = ">=3.10"
readme = "README.md"
license = { text = "MIT" }

dependencies = [
    'tables; sys_platform!="darwin" or platform_machine!="arm64"',
    'tables @ git+https://github.com/PyTables/PyTables.git@v3.9.0 ; sys_platform=="darwin" and platform_machine=="arm64"',
]

[tool.pdm]
distribution = false

Lock for multiple targets:

user@user:~/repos/sandbox/pdm$  pdm lock --platform linux --append \
        && pdm lock --platform windows --append \
        && pdm lock --platform macos_arm64 --append \
        && pdm lock --platform macos_x86_64 --append

Add a dependency:

user@user:~/repos/sandbox/pdm$  pdm add tqdm -v
Adding packages to default dependencies: tqdm
STATUS: Resolving dependencies
STATUS: Resolve for environment (>=3.10, manylinux_2_17_x86_64)
pdm.termui: ======== Start resolving requirements ========
pdm.termui:   tables; sys_platform != "darwin" or platform_machine != "arm64"
pdm.termui:   tables @ git+https://github.com/PyTables/PyTables.git@v3.9.0 ; sys_platform == "darwin" and platform_machine == "arm64"
pdm.termui:   tqdm
pdm.termui:   Adding requirement tables; sys_platform != "darwin" or platform_machine != "arm64"
pdm.termui:   Adding requirement tqdm
pdm.termui:   Adding requirement python>=3.10
pdm.termui: ======== Starting round 0 ========
STATUS: Resolving: new pin python>=3.10
pdm.termui: Pinning: python None
pdm.termui: ======== Ending round 0 ========
pdm.termui: ======== Starting round 1 ========
pdm.termui:   Adding requirement numpy>=1.20.0(from tables 3.10.1)
pdm.termui:   Adding requirement numexpr>=2.6.2(from tables 3.10.1)
pdm.termui:   Adding requirement packaging(from tables 3.10.1)
pdm.termui:   Adding requirement py-cpuinfo(from tables 3.10.1)
pdm.termui:   Adding requirement blosc2>=2.3.0(from tables 3.10.1)
pdm.termui:   Adding requirement typing-extensions>=4.4.0(from tables 3.10.1)
STATUS: Resolving: new pin tables 3.10.1
pdm.termui: Pinning: tables 3.10.1
pdm.termui: ======== Ending round 1 ========
pdm.termui: ======== Starting round 2 ========
STATUS: Resolving: new pin tqdm 4.66.5
pdm.termui: Pinning: tqdm 4.66.5
pdm.termui: ======== Ending round 2 ========
pdm.termui: ======== Starting round 3 ========
pdm.termui:   Adding requirement numpy>=1.21(from blosc2 2.7.1)
pdm.termui:   Adding requirement ndindex>=1.4(from blosc2 2.7.1)
pdm.termui:   Adding requirement msgpack(from blosc2 2.7.1)
pdm.termui:   Adding requirement numexpr(from blosc2 2.7.1)
pdm.termui:   Adding requirement py-cpuinfo(from blosc2 2.7.1)
STATUS: Resolving: new pin blosc2 2.7.1
pdm.termui: Pinning: blosc2 2.7.1
pdm.termui: ======== Ending round 3 ========
pdm.termui: ======== Starting round 4 ========
STATUS: Resolving: new pin numpy 2.1.1
pdm.termui: Pinning: numpy 2.1.1
pdm.termui: ======== Ending round 4 ========
pdm.termui: ======== Starting round 5 ========
pdm.termui:   Adding requirement numpy>=1.23.0(from numexpr 2.10.1)
STATUS: Resolving: new pin numexpr 2.10.1
pdm.termui: Pinning: numexpr 2.10.1
pdm.termui: ======== Ending round 5 ========
pdm.termui: ======== Starting round 6 ========
STATUS: Resolving: new pin typing-extensions 4.12.2
pdm.termui: Pinning: typing-extensions 4.12.2
pdm.termui: ======== Ending round 6 ========
pdm.termui: ======== Starting round 7 ========
STATUS: Resolving: new pin packaging 24.1
pdm.termui: Pinning: packaging 24.1
pdm.termui: ======== Ending round 7 ========
pdm.termui: ======== Starting round 8 ========
STATUS: Resolving: new pin py-cpuinfo 9.0.0
pdm.termui: Pinning: py-cpuinfo 9.0.0
pdm.termui: ======== Ending round 8 ========
pdm.termui: ======== Starting round 9 ========
STATUS: Resolving: new pin ndindex 1.8
pdm.termui: Pinning: ndindex 1.8
pdm.termui: ======== Ending round 9 ========
pdm.termui: ======== Starting round 10 ========
STATUS: Resolving: new pin msgpack 1.0.8
pdm.termui: Pinning: msgpack 1.0.8
pdm.termui: ======== Ending round 10 ========
pdm.termui: ======== Starting round 11 ========
pdm.termui: ======== Resolution Result ========
pdm.termui: Stable pins:
pdm.termui:              python None
pdm.termui:              tables 3.10.1
pdm.termui:                tqdm 4.66.5
pdm.termui:              blosc2 2.7.1
pdm.termui:               numpy 2.1.1
pdm.termui:             numexpr 2.10.1
pdm.termui:   typing-extensions 4.12.2
pdm.termui:           packaging 24.1
pdm.termui:          py-cpuinfo 9.0.0
pdm.termui:             ndindex 1.8
pdm.termui:             msgpack 1.0.8
pdm.termui: Fetching hashes for tqdm@4.66.5
STATUS: Resolve for environment (>=3.10, windows_amd64)
pdm.termui: ======== Start resolving requirements ========
pdm.termui:   tables; sys_platform != "darwin" or platform_machine != "arm64"
pdm.termui:   tables @ git+https://github.com/PyTables/PyTables.git@v3.9.0 ; sys_platform == "darwin" and platform_machine == "arm64"
pdm.termui:   tqdm
pdm.termui:   Adding requirement tables; sys_platform != "darwin" or platform_machine != "arm64"
pdm.termui:   Adding requirement tqdm
pdm.termui:   Adding requirement python>=3.10
pdm.termui: ======== Starting round 0 ========
STATUS: Resolving: new pin python>=3.10
pdm.termui: Pinning: python None
pdm.termui: ======== Ending round 0 ========
pdm.termui: ======== Starting round 1 ========
pdm.termui:   Adding requirement numpy>=1.20.0(from tables 3.10.1)
pdm.termui:   Adding requirement numexpr>=2.6.2(from tables 3.10.1)
pdm.termui:   Adding requirement packaging(from tables 3.10.1)
pdm.termui:   Adding requirement py-cpuinfo(from tables 3.10.1)
pdm.termui:   Adding requirement blosc2>=2.3.0(from tables 3.10.1)
pdm.termui:   Adding requirement typing-extensions>=4.4.0(from tables 3.10.1)
STATUS: Resolving: new pin tables 3.10.1
pdm.termui: Pinning: tables 3.10.1
pdm.termui: ======== Ending round 1 ========
pdm.termui: ======== Starting round 2 ========
pdm.termui:   Adding requirement colorama; platform_system == "Windows"(from tqdm 4.66.5)
STATUS: Resolving: new pin tqdm 4.66.5
pdm.termui: Pinning: tqdm 4.66.5
pdm.termui: ======== Ending round 2 ========
pdm.termui: ======== Starting round 3 ========
pdm.termui:   Adding requirement numpy>=1.21(from blosc2 2.7.1)
pdm.termui:   Adding requirement ndindex>=1.4(from blosc2 2.7.1)
pdm.termui:   Adding requirement msgpack(from blosc2 2.7.1)
pdm.termui:   Adding requirement numexpr(from blosc2 2.7.1)
pdm.termui:   Adding requirement py-cpuinfo(from blosc2 2.7.1)
STATUS: Resolving: new pin blosc2 2.7.1
pdm.termui: Pinning: blosc2 2.7.1
pdm.termui: ======== Ending round 3 ========
pdm.termui: ======== Starting round 4 ========
STATUS: Resolving: new pin numpy 2.1.1
pdm.termui: Pinning: numpy 2.1.1
pdm.termui: ======== Ending round 4 ========
pdm.termui: ======== Starting round 5 ========
pdm.termui:   Adding requirement numpy>=1.23.0(from numexpr 2.10.1)
STATUS: Resolving: new pin numexpr 2.10.1
pdm.termui: Pinning: numexpr 2.10.1
pdm.termui: ======== Ending round 5 ========
pdm.termui: ======== Starting round 6 ========
STATUS: Resolving: new pin typing-extensions 4.12.2
pdm.termui: Pinning: typing-extensions 4.12.2
pdm.termui: ======== Ending round 6 ========
pdm.termui: ======== Starting round 7 ========
STATUS: Resolving: new pin colorama 0.4.6
pdm.termui: Pinning: colorama 0.4.6
pdm.termui: ======== Ending round 7 ========
pdm.termui: ======== Starting round 8 ========
STATUS: Resolving: new pin packaging 24.1
pdm.termui: Pinning: packaging 24.1
pdm.termui: ======== Ending round 8 ========
pdm.termui: ======== Starting round 9 ========
STATUS: Resolving: new pin py-cpuinfo 9.0.0
pdm.termui: Pinning: py-cpuinfo 9.0.0
pdm.termui: ======== Ending round 9 ========
pdm.termui: ======== Starting round 10 ========
STATUS: Resolving: new pin ndindex 1.8
pdm.termui: Pinning: ndindex 1.8
pdm.termui: ======== Ending round 10 ========
pdm.termui: ======== Starting round 11 ========
STATUS: Resolving: new pin msgpack 1.0.8
pdm.termui: Pinning: msgpack 1.0.8
pdm.termui: ======== Ending round 11 ========
pdm.termui: ======== Starting round 12 ========
pdm.termui: ======== Resolution Result ========
pdm.termui: Stable pins:
pdm.termui:              python None
pdm.termui:              tables 3.10.1
pdm.termui:                tqdm 4.66.5
pdm.termui:              blosc2 2.7.1
pdm.termui:               numpy 2.1.1
pdm.termui:             numexpr 2.10.1
pdm.termui:   typing-extensions 4.12.2
pdm.termui:            colorama 0.4.6
pdm.termui:           packaging 24.1
pdm.termui:          py-cpuinfo 9.0.0
pdm.termui:             ndindex 1.8
pdm.termui:             msgpack 1.0.8
pdm.termui: Fetching hashes for tqdm@4.66.5
pdm.termui: Fetching hashes for colorama@0.4.6
STATUS: Resolve for environment (>=3.10, macos_12_0_arm64)
pdm.termui: ======== Start resolving requirements ========
pdm.termui:   tables; sys_platform != "darwin" or platform_machine != "arm64"
pdm.termui:   tables @ git+https://github.com/PyTables/PyTables.git@v3.9.0 ; sys_platform == "darwin" and platform_machine == "arm64"
pdm.termui:   tqdm
pdm.termui:   Adding requirement tables @ git+https://github.com/PyTables/PyTables.git@v3.9.0 ; sys_platform == "darwin" and platform_machine == "arm64"
pdm.termui:   Adding requirement tqdm
pdm.termui:   Adding requirement python>=3.10
pdm.termui: ======== Starting round 0 ========
STATUS: Resolving: new pin python>=3.10
pdm.termui: Pinning: python None
pdm.termui: ======== Ending round 0 ========
pdm.termui: ======== Starting round 1 ========
pdm.termui:   Adding requirement numpy>=1.19.0(from tables 3.9.0)
pdm.termui:   Adding requirement numexpr>=2.6.2(from tables 3.9.0)
pdm.termui:   Adding requirement packaging(from tables 3.9.0)
pdm.termui:   Adding requirement py-cpuinfo(from tables 3.9.0)
pdm.termui:   Adding requirement blosc2>=2.2.8(from tables 3.9.0)
STATUS: Resolving: new pin tables 3.9.0
pdm.termui: Pinning: tables 3.9.0
pdm.termui: ======== Ending round 1 ========
pdm.termui: ======== Starting round 2 ========
STATUS: Resolving: new pin tqdm 4.66.5
pdm.termui: Pinning: tqdm 4.66.5
pdm.termui: ======== Ending round 2 ========
pdm.termui: ======== Starting round 3 ========
pdm.termui:   Adding requirement numpy>=1.21(from blosc2 2.7.1)
pdm.termui:   Adding requirement ndindex>=1.4(from blosc2 2.7.1)
pdm.termui:   Adding requirement msgpack(from blosc2 2.7.1)
pdm.termui:   Adding requirement numexpr(from blosc2 2.7.1)
pdm.termui:   Adding requirement py-cpuinfo(from blosc2 2.7.1)
STATUS: Resolving: new pin blosc2 2.7.1
pdm.termui: Pinning: blosc2 2.7.1
pdm.termui: ======== Ending round 3 ========
pdm.termui: ======== Starting round 4 ========
STATUS: Resolving: new pin numpy 2.1.1
pdm.termui: Pinning: numpy 2.1.1
pdm.termui: ======== Ending round 4 ========
pdm.termui: ======== Starting round 5 ========
pdm.termui:   Adding requirement numpy>=1.23.0(from numexpr 2.10.1)
STATUS: Resolving: new pin numexpr 2.10.1
pdm.termui: Pinning: numexpr 2.10.1
pdm.termui: ======== Ending round 5 ========
pdm.termui: ======== Starting round 6 ========
STATUS: Resolving: new pin packaging 24.1
pdm.termui: Pinning: packaging 24.1
pdm.termui: ======== Ending round 6 ========
pdm.termui: ======== Starting round 7 ========
STATUS: Resolving: new pin py-cpuinfo 9.0.0
pdm.termui: Pinning: py-cpuinfo 9.0.0
pdm.termui: ======== Ending round 7 ========
pdm.termui: ======== Starting round 8 ========
STATUS: Resolving: new pin ndindex 1.8
pdm.termui: Pinning: ndindex 1.8
pdm.termui: ======== Ending round 8 ========
pdm.termui: ======== Starting round 9 ========
STATUS: Resolving: new pin msgpack 1.0.8
pdm.termui: Pinning: msgpack 1.0.8
pdm.termui: ======== Ending round 9 ========
pdm.termui: ======== Starting round 10 ========
pdm.termui: ======== Resolution Result ========
pdm.termui: Stable pins:
pdm.termui:       python None
pdm.termui:       tables git+https://github.com/PyTables/PyTables.git@f85dc2667492d07a8de57bfb14cb718c5400d02d
pdm.termui:         tqdm 4.66.5
pdm.termui:       blosc2 2.7.1
pdm.termui:        numpy 2.1.1
pdm.termui:      numexpr 2.10.1
pdm.termui:    packaging 24.1
pdm.termui:   py-cpuinfo 9.0.0
pdm.termui:      ndindex 1.8
pdm.termui:      msgpack 1.0.8
pdm.termui: Fetching hashes for tqdm@4.66.5
STATUS: Resolve for environment (>=3.10, macos_12_0_x86_64)
pdm.termui: ======== Start resolving requirements ========
pdm.termui:   tables; sys_platform != "darwin" or platform_machine != "arm64"
pdm.termui:   tables @ git+https://github.com/PyTables/PyTables.git@v3.9.0 ; sys_platform == "darwin" and platform_machine == "arm64"
pdm.termui:   tqdm
pdm.termui:   Adding requirement tables; sys_platform != "darwin" or platform_machine != "arm64"
pdm.termui:   Adding requirement tqdm
pdm.termui:   Adding requirement python>=3.10
pdm.termui: ======== Starting round 0 ========
STATUS: Resolving: new pin python>=3.10
pdm.termui: Pinning: python None
pdm.termui: ======== Ending round 0 ========
pdm.termui: ======== Starting round 1 ========
pdm.termui:   Adding requirement numpy>=1.20.0(from tables 3.10.1)
pdm.termui:   Adding requirement numexpr>=2.6.2(from tables 3.10.1)
pdm.termui:   Adding requirement packaging(from tables 3.10.1)
pdm.termui:   Adding requirement py-cpuinfo(from tables 3.10.1)
pdm.termui:   Adding requirement blosc2>=2.3.0(from tables 3.10.1)
pdm.termui:   Adding requirement typing-extensions>=4.4.0(from tables 3.10.1)
STATUS: Resolving: new pin tables 3.10.1
pdm.termui: Pinning: tables 3.10.1
pdm.termui: ======== Ending round 1 ========
pdm.termui: ======== Starting round 2 ========
STATUS: Resolving: new pin tqdm 4.66.5
pdm.termui: Pinning: tqdm 4.66.5
pdm.termui: ======== Ending round 2 ========
pdm.termui: ======== Starting round 3 ========
pdm.termui:   Adding requirement numpy>=1.21(from blosc2 2.7.1)
pdm.termui:   Adding requirement ndindex>=1.4(from blosc2 2.7.1)
pdm.termui:   Adding requirement msgpack(from blosc2 2.7.1)
pdm.termui:   Adding requirement numexpr(from blosc2 2.7.1)
pdm.termui:   Adding requirement py-cpuinfo(from blosc2 2.7.1)
STATUS: Resolving: new pin blosc2 2.7.1
pdm.termui: Pinning: blosc2 2.7.1
pdm.termui: ======== Ending round 3 ========
pdm.termui: ======== Starting round 4 ========
STATUS: Resolving: new pin numpy 2.1.1
pdm.termui: Pinning: numpy 2.1.1
pdm.termui: ======== Ending round 4 ========
pdm.termui: ======== Starting round 5 ========
pdm.termui:   Adding requirement numpy>=1.23.0(from numexpr 2.10.1)
STATUS: Resolving: new pin numexpr 2.10.1
pdm.termui: Pinning: numexpr 2.10.1
pdm.termui: ======== Ending round 5 ========
pdm.termui: ======== Starting round 6 ========
STATUS: Resolving: new pin typing-extensions 4.12.2
pdm.termui: Pinning: typing-extensions 4.12.2
pdm.termui: ======== Ending round 6 ========
pdm.termui: ======== Starting round 7 ========
STATUS: Resolving: new pin packaging 24.1
pdm.termui: Pinning: packaging 24.1
pdm.termui: ======== Ending round 7 ========
pdm.termui: ======== Starting round 8 ========
STATUS: Resolving: new pin py-cpuinfo 9.0.0
pdm.termui: Pinning: py-cpuinfo 9.0.0
pdm.termui: ======== Ending round 8 ========
pdm.termui: ======== Starting round 9 ========
STATUS: Resolving: new pin ndindex 1.8
pdm.termui: Pinning: ndindex 1.8
pdm.termui: ======== Ending round 9 ========
pdm.termui: ======== Starting round 10 ========
STATUS: Resolving: new pin msgpack 1.0.8
pdm.termui: Pinning: msgpack 1.0.8
pdm.termui: ======== Ending round 10 ========
pdm.termui: ======== Starting round 11 ========
pdm.termui: ======== Resolution Result ========
pdm.termui: Stable pins:
pdm.termui:              python None
pdm.termui:              tables 3.10.1
pdm.termui:                tqdm 4.66.5
pdm.termui:              blosc2 2.7.1
pdm.termui:               numpy 2.1.1
pdm.termui:             numexpr 2.10.1
pdm.termui:   typing-extensions 4.12.2
pdm.termui:           packaging 24.1
pdm.termui:          py-cpuinfo 9.0.0
pdm.termui:             ndindex 1.8
pdm.termui:             msgpack 1.0.8
pdm.termui: Fetching hashes for tqdm@4.66.5
🔒 Lock successful.
Changes are written to pyproject.toml.
STATUS: Resolving packages from lockfile...
All packages are synced to date, nothing to do.

🎉 All complete!

Actual behavior

Looking into pdm.lock, all packages (except for newly added ones) receive an incorrect marker: marker = "sys_platform != \"darwin\" or platform_machine != \"arm64\"" which they didn't have before the pdm add command.

Expected behavior

Locked packages to not receive new markers.

Environment Information

user@user:~/repos/sandbox/pdm$ pdm info && pdm info --env
PDM version:
  2.17.1
Python Interpreter:
  /home/user/repos/sandbox/pdm/.venv/bin/python (3.12)
Project Root:
  /home/user/repos/sandbox/pdm
Local Packages:

INFO: PDM 2.17.1 is installed, while 2.18.1 is available.
Please run `pdm self update` to upgrade.
Run `pdm config check_update false` to disable the check.
{
  "implementation_name": "cpython",
  "implementation_version": "3.10.14",
  "os_name": "posix",
  "platform_machine": "x86_64",
  "platform_release": "6.8.0-41-generic",
  "platform_system": "Linux",
  "platform_version": "#41-Ubuntu SMP PREEMPT_DYNAMIC Fri Aug  2 20:41:06 UTC 2024",
  "python_full_version": "3.12.3",
  "platform_python_implementation": "CPython",
  "python_version": "3.12",
  "sys_platform": "linux"
}
frostming commented 1 month ago

I can't reproduce with the following steps:

[project]
name = "test_pdm"
version = "0.1.0"
description = "Default template for PDM package"
authors = [
    {name = "me", email = "me@example.com"},
]
dependencies = [
    'tables; sys_platform!="darwin" or platform_machine!="arm64"',
    'tables @ git+https://github.com/PyTables/PyTables.git@v3.9.0 ; sys_platform=="darwin" and platform_machine=="arm64"',
]
requires-python = ">=3.10"
readme = "README.md"
license = {text = "MIT"}

[tool.pdm]
distribution = false
  1. pdm lock --platform linux \
    && pdm lock --platform windows --append \
    && pdm lock --platform macos_arm64 --append \
    && pdm lock --platform macos_x86_64 --append
  2. Result: pdm.lock.old.txt
  3. pdm add tqdm --no-sync
  4. Result: pdm.lock.txt
MhdMartini commented 1 month ago

I ran the experiment here and you're right, nothing weird happens. I need to find out is why it happens on my local machine.. But no bug! Thanks for responding.. Cheers!

sanmai-NL commented 2 weeks ago

@MhdMartini Can you close this then?