Open q-aaronzolnailucas opened 8 months ago
~1.2.0 doesn't work for poetry install, with or without allow-prereleases
correct, because 1.2.0.dev
comes before 1.2.0
: so 1.2.0.dev
does not satisfy that requirement
I find it hard to follow what you are trying to achieve here but I will take a guess that it is a better fit for a plugin than something that would make sense in poetry proper.
Maybe prototype it out yourself and if you are convinced that it belongs, submit a pull request.
@dimbleby shouldn't allow-prereleases = true
fix that?
Effectively, the feature I'm requesting is to inject dependency versions at release time. Since poetry has steered clear of using environment variables (#208 #481 ) this would be a neat way to hit one of the usecases that was asked for on those threads.
I'm not going to have capacity or expertise to develop this myself, as a plugin or otherwise.
shouldn't allow-prereleases = true fix that?
allow-prereleases = true
would allow prereleases that do satisfy the requirement (eg 1.3.0.dev
): but a version has to satisfy the requirement first.
injecting dependency versions at release time does not make any sense to me - how do you even know that there is a solution?
if you are not going to develop this then I think it is very likely that no-one else is either.
injecting dependency versions at release time does not make any sense to me
Well, I'm not the first to ask and it's standard practice for multiproject releases of other languages.
If you have 2 packages in the same repo, A and B, on the same release cycle, and B depends on A, then you can't release B v1.0 until you've released A v1.0. Currently, steps needed here are:
It's not great that this pipeline causes such a delta in these files during excecution and is so sensitive to order. Manually writing CI pipelines like this is tiresome. Imagine this happened with a more complicated dependency graph, or with a circular dependency - which is possible. Instead, why not allow dependency injection for packages I'm releasing.
Here's a quick plugin that lets me use env vars from .env
files or otherwise which is now my workaround:
import os.path
from typing import TYPE_CHECKING, Any, Dict
import poetry.core.pyproject.toml
from dotenv import load_dotenv
from poetry.plugins import Plugin
if TYPE_CHECKING:
from cleo.io.io import IO
from poetry.poetry import Poetry
def read_pyproject_toml(self: poetry.core.pyproject.toml.PyProjectTOML) -> Dict[str, Any]:
"""Patched version of poetry-core's PyProjectTOML.data."""
if self._data is None:
if not self.path.exists():
self._data = {}
else:
with self.path.open("rb") as f:
load_dotenv()
content = os.path.expandvars(f.read().decode("utf-8"))
self._data = poetry.core.pyproject.toml.tomllib.loads(content)
return self._data
class EnvVarPoetryPlugin(Plugin):
"""Poetry plugin to patch the toml parser to interpolate environment variables."""
def activate(self, _: "Poetry", io: "IO") -> None:
"""Activate the plugin."""
if io.is_debug():
io.write_line("<debug>Patching toml parser to interpolate environment variables</debug>")
self.patch_toml_parse_expandvars()
@staticmethod
def patch_toml_parse_expandvars() -> None:
"""Patch the PyProjectTOML.data property to expand environment variables."""
poetry.core.pyproject.toml.PyProjectTOML.data = property(read_pyproject_toml) # type: ignore
You want to publish a B that depends on A 1.0, before A 1.0 even exists?
IMO this is clearly something that poetry should not support, poetry is all about ensuring that dependencies are consistent and can be resolved.
Exploring a plug-in based approach for this dangerous operation sounds like exactly the right way to go. I think it unlikely that this function will make it into poetry proper. Anyway I stand by the advice that if you don't do it yourself then probably no-one else will either.
You want to publish a B that depends on A 1.0, before A 1.0 even exists? No, I want to publish them at the same time without CI knowing about the dependency graph.
Thanks for the helpful advice anyway - I'll keep the community posted if we develop on this.
This is what the Poetry MonoRepo Dependency Plugin Does
Mix it with the Poetry Dynamic Versioning plugin and you can get a new dynamic version for your monorepo packages.
So you build all of your packages they'll all receive the dynamic version based on the git tag and then push them to the pypi repo- You'll then be able to install any of them and their dependencies will be available.
Issue Kind
Change in current behaviour
Description
Support injecting dependency versions during publish/build for multi-project releases.
My usecase
I have a multi-project, single repo situation. The separate projects share a single venv and a top level
pyproject.toml
that has dev dependencies and configuration for linters/cqa - so that these standards are shared.the top-level pyproject.toml has ./pyproject.toml
That's all well and good, but
product-two
depends onproduct-one
. We manage this by using the same release cycle for both. ./product-one/pyproject.toml./product-two/pyproject.toml
Now, using
~1.2.0
doesn't work forpoetry install
, with or withoutallow-prereleases
. Specifying "1.2.0-dev" does work, but we'd like to allow different patch/build version differences just to be flexible, and not have to set the version in so many places. The best would be to leave this version as*
everywhere, and then have a dependency version inject parameter at build time. This would play nice with versioning plugins too.Example implementation:
pyproject.toml
then publish:
This could have the caveat that it only works for path and git dependencies.
Impact
Better multi-project release processes
Workarounds
using sed to fix versions before release pipelines run, modifying build artifacts