search

python-poetry / poetry

Python packaging and dependency management made easy
https://python-poetry.org
MIT License
31.71k stars 2.27k forks source link

Version Matching - Local Identifiers #6570

Closed bradleyhurley closed 2 years ago

bradleyhurley commented 2 years ago

Issue

I believe this issue is similar to https://github.com/python-poetry/poetry/issues/4729 but didn't consider it a duplicate because the issue is marked as closed.

This issue might be fixed up: https://github.com/python-poetry/poetry-core/pull/379

from poetry.core.semver.helpers import parse_constraint

dev_build = parse_constraint('==0.92.0+f4f0e5bc')
release_build = parse_constraint('==0.92.0')

release_build.allows(dev_build)
True

dev_build.allows(release_build)
False

release_build.next_breaking()
<Version 0.93.0>

dev_build.next_breaking()
<Version 0.93.0>

On version 1.1.13 adding a dependency to the pyproject.toml would always result in that specific version being written to the poetry.lock when running poetry lock --no-update.

After upgrading to 1.2.1 we noticed that adding a pinned version of an internal library to the pyproject.toml file and running poetry lock --no-update would result in the lock file containing a pointer to the expected version, but it would include the local identifier.

The poetry.lock would look like the example below.

[[package]]
name = "other-internal-project"
version = "0.92.0+f4f0e5bc"
description = ""
category = "main"
optional = false

A workaround that we found was adding the source to the pyproject.toml. Package versions with local identifiers are published to the local-pypi-snapshot PyPI instance and versions without are published to local-pypi-release.

other-internal-project" = {version = "0.92.0", source = "local-pypi-release"}
neersighted commented 2 years ago

You might need to be more specific about the issue you're running into -- at least I am having a hard time following. It sounds like the lock file is recording a garbage version -- could you please post the expected lock file entry and the actual lock file entry, as well as describe concrete versions so people can understand where they are incorrect?

bradleyhurley commented 2 years ago

Thanks, and sorry about that.

other-internal-project == 0.92.0+f4f0e5bc is a valid package name and version deployed to local-pypi-snapshot other-internal-project == 0.92.0 is a valid package name and version deployed to local-pypi-release

When adding other-internal-project = "0.92.0" to the pyproject.toml as shown in the gist we end up with the local identifier version in the lock file as shown below after running poetry lock --no-update.

Actual Lock File Entry

[[package]]
name = "other-internal-project"
version = "0.92.0+f4f0e5bc"
description = ""
category = "main"
optional = false

Expected Lock File Entry

[[package]]
name = "other-internal-project"
version = "0.92.0"
description = ""
category = "main"
optional = false

We are using the short git commit hash for development builds.

neersighted commented 2 years ago

I don't think this is a bug -- they are the same version and according to PEP 440/Poetry's internal logic, are treated as equivalent when you ask for a global version. Local versions are really just informational tags appended to the end of a version in other words -- you can request them specifically, but when you ask for a global version, they are not different versions as far as satisfying version constraints is concerned. Their intended use is to signify that they may contain code different from an 'upstream' version -- in this case Poetry considers them interchangeable as both versions claim to provide '0.92.0'.

You likely want to tag your builds as prerelease if you are providing development builds, e.g. 0.92.0.dev0+f4f0e5bc, and then increment the devX number as you provide additional prerelease builds (if you don't want to do tagged alphas/betas or similar).

I think we've done our best to call this out in the docs, but maybe it could be made clearer -- feedback on where you looked and how we can make this digestible would be welcome.

bradleyhurley commented 2 years ago

If the specified version identifier is a public version identifier (no local version label), then the local version label of any candidate versions MUST be ignored when matching versions.

If the specified version identifier is a local version identifier, then the local version labels of candidate versions MUST be considered when matching versions, with the public version identifier being matched as described above, and the local version label being checked for equivalence using a strict string equality comparison.

https://www.python.org/dev/peps/pep-0440/#version-matching

Doesn’t the version matching section of PEP440 state they should be treated as different versions?

The examples in issue #4729 that is indicated as a bug seem to also indicate the versions should be treated as distinct.

neersighted commented 2 years ago

That is correct in the local -> global direction, but as I understand your issue (you're getting local versions when you don't request them), you're talking global -> local. If you request X.Y.Z, X.Y.Z+local is a valid way to provide it. I probably should have been more specific in my answer that I am only talking about the specific case of not specifying a local identifier.

github-actions[bot] commented 8 months ago

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.