`install-poetry.py`: Use for `1.1.9` and up

[tony]

Part 1 of 2 fix for #10 / https://github.com/python-poetry/poetry/issues/3345

Error: ModuleNotFoundError: No module named 'cleo', Versions: Python 3.10 and poetry 1.1.9 - 1.1.11 during poetry invocation 'Link' object has no attribute 'is_absolute' poetry 1.1.7 - 1.1.8 during poetry install of pycparser==0.20.0

See also:

• https://github.com/python-poetry/poetry/releases/tag/1.1.7
• https://github.com/asdf-community/asdf-poetry/issues/10#issuecomment-979310607
• https://github.com/python-poetry/poetry/issues/4409#issuecomment-922277122

[tony]

@crflynn Are either / both of these (including #15) worth merging?

14 - expands install-poetry.py from poetry 2.0 -> 1.1.9

15 - uses new official install location for install-poetry.py

I'm using both of them in a fork and it's working well for me.

[crflynn]

I'm testing some changes locally starting with this PR.

ModuleNotFoundError: No module named 'cleo', Versions: Python 3.10 and poetry 1.1.9 - 1.1.11

I'm able to repro this, and while it does fix the cleo error, it also expands issue #10 for ^1.1.9 on python <3.10.

'Link' object has no attribute 'is_absolute' poetry 1.1.7 - 1.1.8

Not able to repro this. Is there a specific python version I should be using/installing with?

[tony]

@crflynn Taking a look

[tony]

@crflynn

'Link' object has no attribute 'is_absolute' poetry 1.1.7 - 1.1.8 Not able to repro this. Is there a specific python version I should be using/installing with?

Happened with with pyparser==2.20

I can also set it to 1.1.7 in this PR if it was just me

❯ poetry install
Creating virtualenv myproject in /home/t/work/project/myproject/.venv
Installing dependencies from lock file

Package operations: 229 installs, 0 updates, 0 removals

  • Installing pycparser (2.20): Failed

  AttributeError

  'Link' object has no attribute 'is_absolute'

  at ~/.config/.asdf/installs/poetry/1.1.7/venv/lib/python3.7/site-packages/poetry/core/packages/file_dependency.py:33 in __init__
       29│         self._path = path
       30│         self._base = base or Path.cwd()
       31│         self._full_path = path
       32│
    →  33│         if not self._path.is_absolute():
       34│             try:
       35│                 self._full_path = self._base.joinpath(self._path).resolve()
       36│             except FileNotFoundError:
       37│                 raise ValueError("Directory {} does not exist".format(self._path))

You can reproduce with this pyproject.toml:

[tool.poetry]
name = "my project"
version = "1.0.0"
description = "None"
authors = ["Anonymous"]

[tool.poetry.dependencies]
python = "==3.*.*,>=3.7.0"
pycparser = "==2.20"

The bug happens here: https://github.com/python-poetry/poetry/issues/4409#issuecomment-903173358

[tony]

@crflynn Rebased with #15

Any preference on whether to stick with 1.1.9 or go to 1.1.7?

[crflynn]

I'm reluctant to merge this because:

while it does fix the cleo error, it also expands issue #10 for ^1.1.9 on python <3.10.

and I would rather not further break the functionality of this plugin on what I would guess is currently a more common situation for users than ^1.1.9 on python=3.10.

You point out some discussion here https://github.com/asdf-community/asdf-poetry/pull/15#issuecomment-980473765 which seems to be related to #10 and indicates an incoming fix in poetry 1.2. Since we have 1.2+ on the new installer perhaps it's better wait for that release and inform users that in order to use python 3.10 they must be using at least poetry 1.2.

[tony]

The plugin is broke 1.1.9 - 1.1.11 with python 3.10 currently, this is a patch to add support for those systems.

Since we have 1.2+ on the new installer perhaps it's better wait for that release and inform users that in order to use python 3.10 they must be using at least poetry 1.2.

Can you rephrase? Is it a plugins place to tell the user / developers what python version and poetry version to use?

1.1.9 - 1.1.11 is broke for me without this patch. I am not sure how this would further break functionality.

1.1.9 - As-is:

``` asdf install poetry 1.1.9 poRetrieving Poetry metadata et This installer is deprecated. Poetry versions installed using this script will not be able to use 'self update' command to upgrade to 1.2.0a1 or later. # Welcome to Poetry! This will download and install the latest version of Poetry, a dependency and package manager for Python. It will add the `poetry` command to Poetry's bin directory, located at: $HOME/.config/.asdf/installs/poetry/1.1.9/bin This path needs to be in your `PATH` environment variable, but will not be added automatically. You can uninstall at any time by executing this script with the --uninstall option, and these changes will be reverted. Installing version: 1.1.9 r - Downloading poetry-1.1.9-linux.tar.gz (62.07MB) Poetry (1.1.9) is installed now. Great! To get started you need Poetry's bin directory ($HOME/.config/.asdf/installs/poetry/1.1.9/bin) in your `PATH` environment variable. To configure your current shell run `source $HOME/.config/.asdf/installs/poetry/1.1.9/env` project on  python-3.10 [!] ❯ poetry Traceback (most recent call last): File "~/.config/.asdf/installs/poetry/1.1.9/bin/poetry", line 17, in from poetry.console import main File "~/.config/.asdf/installs/poetry/1.1.9/lib/poetry/console/__init__.py", line 1, in from .application import Application File "~/.config/.asdf/installs/poetry/1.1.9/lib/poetry/console/application.py", line 3, in from cleo import Application as BaseApplication ModuleNotFoundError: No module named 'cleo' ```

1.1.9 - WIth PR:

``` asdf install poetry 1.1.9 Retrieving Poetry metadata # Welcome to Poetry! This will download and install the latest version of Poetry, a dependency and package manager for Python. It will add the `poetry` command to Poetry's bin directory, located at: ~/.config/.asdf/installs/poetry/1.1.9/bin You can uninstall at any time by executing this script with the --uninstall option, and these changes will be reverted. Installing Poetry (1.1.9): Done Poetry (1.1.9) is installed now. Great! To get started you need Poetry's bin directory (~/.config/.asdf/installs/poetry/1.1.9/bin) in your `PATH` environment variable. Add `export PATH="~/.config/.asdf/installs/poetry/1.1.9/bin:$PATH"` to your shell configuration file. Alternatively, you can call Poetry explicitly with `~/.config/.asdf/installs/poetry/1.1.9/bin/poetry`. You can test that everything is set up by executing: `poetry --version` project on  python-3.10 [!] ❯ poetry Poetry version 1.1.9 USAGE poetry [-h] [-q] [-v [<...>]] [-V] [--ansi] [--no-ansi] [-n] [] ... [] ARGUMENTS The command to execute The arguments of the command GLOBAL OPTIONS -h (--help) Display this help message -q (--quiet) Do not output any message -v (--verbose) Increase the verbosity of messages: "-v" for normal output, "-vv" for more verbose output and "-vvv" for debug -V (--version) Display this application version --ansi Force ANSI output --no-ansi Disable ANSI output -n (--no-interaction) Do not ask any interactive question AVAILABLE COMMANDS about Shows information about Poetry. add Adds a new dependency to pyproject.toml. build Builds a package, as a tarball and a wheel by default. cache Interact with Poetry's cache check Checks the validity of the pyproject.toml file. config Manages configuration settings. debug Debug various elements of Poetry. env Interact with Poetry's project environments. export Exports the lock file to alternative formats. help Display the manual of a command init Creates a basic pyproject.toml file in the current directory. install Installs the project dependencies. lock Locks the project dependencies. new Creates a new Python project at . publish Publishes a package to a remote repository. remove Removes a package from the project dependencies. run Runs a command in the appropriate environment. search Searches for packages on remote repositories. self Interact with Poetry directly. shell Spawns a shell within the virtual environment. show Shows information about packages. update Update the dependencies as according to the pyproject.toml file. version Shows the version of the project or bumps it when a valid bump rule is provided. ```

1.1.11 - Without this patch (as-is):

``` asdf unisntall 1.1.11 asdf install 1.1.11 Traceback (most recent call last): File "~/.config/.asdf/installs/poetry/1.1.11/bin/poetry", line 17, in from poetry.console import main File "~/.config/.asdf/installs/poetry/1.1.11/lib/poetry/console/__init__.py", line 1, in from .application import Application File "~/.config/.asdf/installs/poetry/1.1.11/lib/poetry/console/application.py", line 3, in from cleo import Application as BaseApplication ModuleNotFoundError: No module named 'cleo' ```

1.11 - With this patch

``` asdf uninstall poetry 1.1.11 asdf install poetry 1.1.9 Poetry version 1.1.11 USAGE poetry [-h] [-q] [-v [<...>]] [-V] [--ansi] [--no-ansi] [-n] [] ... [] ARGUMENTS The command to execute The arguments of the command GLOBAL OPTIONS -h (--help) Display this help message -q (--quiet) Do not output any message -v (--verbose) Increase the verbosity of messages: "-v" for normal output, "-vv" for more verbose output and "-vvv" for debug -V (--version) Display this application version --ansi Force ANSI output --no-ansi Disable ANSI output -n (--no-interaction) Do not ask any interactive question AVAILABLE COMMANDS about Shows information about Poetry. add Adds a new dependency to pyproject.toml. build Builds a package, as a tarball and a wheel by default. cache Interact with Poetry's cache check Checks the validity of the pyproject.toml file. config Manages configuration settings. debug Debug various elements of Poetry. env Interact with Poetry's project environments. export Exports the lock file to alternative formats. help Display the manual of a command init Creates a basic pyproject.toml file in the current directory. install Installs the project dependencies. lock Locks the project dependencies. new Creates a new Python project at . publish Publishes a package to a remote repository. remove Removes a package from the project dependencies. run Runs a command in the appropriate environment. search Searches for packages on remote repositories. self Interact with Poetry directly. shell Spawns a shell within the virtual environment. show Shows information about packages. update Update the dependencies as according to the pyproject.toml file. version Shows the version of the project or bumps it when a valid bump rule is provided. ```

[crflynn]

I think you are misunderstanding that issue #10 has nothing to do with the cleo bug. Rather it has to do with poetry not using the python version specified by asdf when creating venvs if it was installed with the new installer, essentially breaking the expected behavior of asdf with python in this particular case. I understand that this fixes the cleo bug with 3.10. However it makes #10 worse because now the issue applies to ^1.1.9 rather than just ^1.2.0.

[tony]

My mistake, I misunderstand / don't understand the #10 issue.

@Kurt-von-Laven you were on the threads relating to the other bug (with virtualenv versions) at https://github.com/python-poetry/poetry/issues/1888#issuecomment-954168590, https://github.com/python-poetry/poetry/issues/4199#issuecomment-956542985.

Do you have any preferences or ideas? If you try this PR / approach how does it work for you in terms of the virtualenv + python interpreter issue?

[crflynn]

To summarize briefly, when poetry is installed with the new installer: if you set your asdf python version to A, but poetry was installed using version B, poetry will disregard your asdf python version (A) and only use version B when creating virtualenvs. @Kurt-von-Laven 's workaround requires manually overriding poetry env config per project.

[Kurt-von-Laven]

I think you are misunderstanding that issue #10 has nothing to do with the cleo bug. Rather it has to do with poetry not using the python version specified by asdf when creating venvs if it was installed with the new installer, essentially breaking the expected behavior of asdf with python in this particular case. I understand that this fixes the cleo bug with 3.10. However it makes #10 worse because now the issue applies to ^1.1.9 rather than just ^1.2.0.

In case this is helpful context, I encountered the cleo bug when attempting to upgrade to Poetry 3.10 and was advised to reinstall Poetry using the version of install-poetry.py on master (after uninstalling all versions of Poetry), which resolved the issue for me.

To summarize briefly, when poetry is installed with the new installer: if you set your asdf python version to A, but poetry was installed using version B, poetry will disregard your asdf python version (A) and only use version B when creating virtualenvs. @Kurt-von-Laven 's workaround requires manually overriding poetry env config per project.

This matches my understanding. I can't claim credit for the workaround, and I've lost track of who first posted it at this point, but I have used it enough times that I'm confident it addresses the issue of Poetry installed via install-poetry.py disregarding your asdf Python version when creating virtualenvs. Some nuances are that the version of Poetry presently on master does not exhibit this bug; you may have to uninstall all versions of Poetry installed on your computer by any means whatsoever in order to experience said fix according to a maintainer of Poetry; that install-poetry.py isn't tied to a Poetry release, although it first appeared during Poetry 1.2 development; and the version of install-poetry.py at the new install URL is functionally equivalent (although not byte-for-byte identical) to the version presently on master. As for what to do about all of this, I stand by my initial recommendation:

The maintainers of Poetry recently pointed out (c.f., #3345) that they have since resolved this issue on master, but the fix (#4433) has not yet been released. Look for it on poetry > 1.2.0a2 once that is released. At that time, I believe this plugin will be able to stop using the deprecated get-poetry.py even for poetry < 1.2. ~ #10

I would either simply keep using the poetry env use "$ASDF_DIR/shims/python or poetry env use "$(asdf which python)" workaround or remain on Poetry 1.1 (which, after all, is still the latest stable release of Poetry) in the meantime. The simplest, most helpful thing this plugin could do right now IMO is explain the needed workaround when installing the versions of Poetry that require it. If a more immediate patch is desired, the comment I linked above shows how to use the version of install-poetry.py on master (or, implicitly, how to pin a specific commit). The latest version of the installer could be copied directly into this project since they are both MIT licensed. At this time, I couldn't say whether these more aggressive solutions would solve more problems than they create, and I am not advocating for them. If anyone needs a fix out in a hurry, probably the most helpful thing they could do is offer their services towards getting the next Poetry release candidate cut.

[tony]

@Kurt-von-Laven If this PR was scoped to python 3.10 (and beyond?), would that be sufficient? (since python 3.10 is DOA on installation with get-poetry.py method)

Assuming the detection of python version wasn't reliable for some reason (python shims for instance) where do you stand on this PR as is?

[Kurt-von-Laven]

@tony, that is a good point; thanks for bringing that up. When I approved this PR I was under the mistaken impression that all of the needed patches were to install-poetry.py itself, but unfortunately that is not the case, so I would continue to use get-poetry.py on Poetry 1.1 for now. It would be nice to issue a warning that Poetry 1.2 is currently required for Python 3.10, however, to spare a lot of people some debugging/hassle with minimal effort. Please correct me if you are seeing something I am missing, but I don't see how this plugin could determine the pertinent Python version at Poetry install time in a meaningful way. The user might manage Python with asdf or pyenv or some other tool. If they use asdf for Python, we don't know which .tool-versions files they have on their machine at Poetry install time. Theoretically, they might have the entire cross-product of all Python and Poetry versions represented across a huge directory hierarchy, and the Python version that happens to be active when Poetry is installed doesn't really have anything to do with how the tools will be used in any deep sense.

[tony]

@Kurt-von-Laven It would be helpful to have something more succinct, if possible.

It would be nice to issue a warning that Poetry 1.2 is currently required for Python 3.10

I'm using it right now on multiple systems with 10+ large projects with different python versions. poetry 1.1.11 (at least) is compatible with 3.10

the plugin is crashing outright and this PR fixes it. It adds 3.10 support for Python 1.1.9 - 1.1.11.

[tony]

@Kurt-von-Laven The reason why is right now what we have threads that I'm having difficulty wrapping my head around. Would @crflynn's summarization at https://github.com/asdf-community/asdf-poetry/pull/14#issuecomment-980487423 describe what you're experiencing?

[Kurt-von-Laven]

@Kurt-von-Laven It would be helpful to have something more succinct, if possible.

Until another version of Poetry 1.2 is released, I would keep the asdf Poetry plugin as is but add some warnings about needing Poetry 1.2 for Python 3.10 and using poetry env use "$(asdf which python)" if you manage Python with asdf to ensure the correct version of Python is used in your virtualenv.

I'm using it right now on multiple systems with 10+ large projects with different python versions. poetry 1.1.11 (at least) is compatible with 3.10.

Right, so my understanding is that it's really any version of Poetry (I think) installed via the install-poetry.py from the new install URL that works with Python 3.10. Hence, because you patched this plugin to use install-poetry.py, you are able to use Python 3.10 with at least some versions of Poetry 1.1.

[tony]

@Kurt-Von-Laven Thank you!

Until another version of Poetry 1.2 is released, I would keep the asdf Poetry plugin as is but add some warnings about needing Poetry 1.2 for Python 3.10 and using poetry env use "$(asdf which python)" if you manage Python with asdf to ensure the correct version of Python is used in your virtualenv.

So users with poetry 1.1 be able to use python 3.10?

As another option: perhaps a --classic flag, or environmental variable to point to the installer?

[Kurt-von-Laven]

@Kurt-von-Laven Thank you!

Until another version of Poetry 1.2 is released, I would keep the asdf Poetry plugin as is but add some warnings about needing Poetry 1.2 for Python 3.10 and using poetry env use "$(asdf which python)" if you manage Python with asdf to ensure the correct version of Python is used in your virtualenv.

So users with poetry 1.1 be able to use python 3.10?

Probably only those that read this thread for now unfortunately? I think the warning should be clear enough about the fact that those users can upgrade to Poetry 1.2 or downgrade to Python 3.9 though. I agree that it's a regrettable situation. I suppose part of the beauty of asdf is that if someone really must use Python 3.10 with Poetry 1.1, they can use your (or their own) fork of the asdf Poetry plugin instead.

As another option: perhaps a --classic flag, or environmental variable to point to the installer?

I am quite new to the asdf ecosystem, but I'm not aware of other plugins that take special options like this, and to my mind one of the principal selling points of asdf is that it exposes the same interface to version management of so many different tools. It's probably better that someone with more experience with asdf weigh in though.

[tony]

Thank you!

I am quite new to the asdf ecosystem, but I'm not aware of other plugins that take special options like this, and to my mind one of the principal selling points of asdf is that it exposes the same interface to version management of so many different tools. It's probably better that someone with more experience with asdf weigh in though.

I went with something asdf friendly: #16

It uses an optional, environmental variable: ASDF_POETRY_INSTALL_URL

For a common plugin example: asdf-python's environmental variable e.g. ASDF_PYTHON_PATCH_URL and asdf-nodejs's NODEJS_CHECK_SIGNATURES

[Kurt-von-Laven]

That is pretty compelling to me at least.

[tony]

This was superseded by #16, ASDF_POETRY_INSTALL_URL: Optionally override installer URL, merged in 29ca741