Using poetry to install dependency with poetry-dynamic-versioning

[rusnyder]

First off - incredible project! Delighted to stumble across it, and a potential godsend for some versioning issues my team has been having.

Issue

I may just be missing something from the docs, but this plugin works flawlessly in all contexts I need it except when trying to install a project that uses poetry-dynamic-versioning via VCS (git).

Using your https://github.com/mtkennerly/playground project (thank you!), pip install works as advertised, but poetry install is a little wonky

Pip

$ python3 -m venv venv
$ . ./venv/bin/activate
$ pip install git+ssh://git@github.com/mtkennerly/playground@master
...
Successfully installed dummy-python-project-0.1.0.post8.dev0+d3a036f

Poetry

pyproject.toml

[tool.poetry]
name = "fake-project"
version = "0.1.0"
description = ""
authors = ["Russell Snyder <russ@arceo.ai>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.8"

dummy-python-project = { git = "https://github.com/mtkennerly/playground" }

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

# First install shows the `version` from pyproject.toml, not the dynamic version
$ poetry install
Creating virtualenv fake-project-GaTXRtqa-py3.8 in /Users/.../Library/Caches/pypoetry-arm64/virtualenvs
Installing dependencies from lock file

Package operations: 1 install, 0 updates, 0 removals

  • Installing dummy-python-project (0.0.999 d3a036f)

# All following installs will "downgrade" to the same version, half-detecting the right version
$ poetry install
Installing dependencies from lock file

Package operations: 0 installs, 1 update, 0 removals

  • Downgrading dummy-python-project (0.1.0.post8.dev0+d3a036f d3a036f -> 0.0.999 d3a036f)

This almost works great, and you can tell under the hood the right version is being detected and reported, but it seems like poetry's locking in-particular is failing to register the version.

Question/Thoughts

Main question: Is this a limitation on the poetry end? It reads like it could be a symptom of the caveat from your README (that a few closed issues also allude to):

The dynamic version is not available during poetry run or poetry shell because of a Poetry design choice that prevents the plugin from cleaning up after itself.

I'm hoping I'm just being dense and missed something simple. I may dig into this in a week or so, but figured I'd post the inquiry in case it was something quick!

[mtkennerly]

Hi! Thanks for the kind words :)

I think this is just a cosmetic issue in poetry install, presumably because it reports the static version in pyproject.toml first before running the build code that updates the version. The right version will still be present in the venv managed by Poetry:

$ poetry install
Creating virtualenv fake-project in C:\tmp\issue154\.venv
Updating dependencies
Resolving dependencies... (1.2s)

Package operations: 1 install, 0 updates, 0 removals

  • Installing dummy-python-project (0.0.999 d3a036f)

$ . .venv/Scripts/activate

$ pip list | grep dummy
dummy-python-project 0.1.0.post8.dev0+d3a036f

[rusnyder]

@mtkennerly Holy cow what a fast reply!

I think this is just a cosmetic issue in poetry install, presumably because it reports the static version in pyproject.toml first before running the build code that updates the version. The right version will still be present in the venv managed by Poetry

You're absolutely correct in that ultimately the right version is exposed both by pip (from your example) and programmatically:

$ poetry run python -c 'import pkg_resources; print(pkg_resources.get_distribution("dummy-python-project").version)'
0.1.0.post8.dev0+d3a036f

The issue is that subsequent poetry install calls think it's a version change, and will "downgrade" (i.e. - reinstall) the package (which isn't the end of the world, but for some of the packages I'm using in a real project can add 20-30 seconds to even full-poetry-cache-hit builds):

$ poetry install -vvv
Using virtualenv: /Users/rusnyder/Library/Caches/pypoetry-arm64/virtualenvs/fake-project-GaTXRtqa-py3.8
Installing dependencies from lock file

Finding the necessary packages for the current system

Package operations: 0 installs, 1 update, 0 removals

  • Downgrading dummy-python-project (0.1.0.post8.dev0+d3a036f d3a036f -> 0.0.999 d3a036f): Pending...
  • Downgrading dummy-python-project (0.1.0.post8.dev0+d3a036f d3a036f -> 0.0.999 d3a036f): Installing...
  • Downgrading dummy-python-project (0.1.0.post8.dev0+d3a036f d3a036f -> 0.0.999 d3a036f)

Ideally subsequent poetry install calls would recognize the dynamic version of the installed package and not reinstall it.

I don't imagine this is an issue with this plugin, rather with poetry itself, so if you don't have any intuition around what might be happening I can do a little digging, just figured I'd check here first!

[mtkennerly]

for some of the packages I'm using in a real project can add 20-30 seconds to even full-poetry-cache-hit builds

Ah, that's true, the extra time is unfortunate.

I can't think of a good solution to this offhand, but if you do investigate further, I'd be interested to hear what you come up with!

[rusnyder]

Sounds good! I'll go ahead and close this for now, and circle back with either an "AHA!" or "Bummer..." when I get back around to digging in.

Thanks for taking a look!