poetry install --sync fails if a previously installed local editable package has been removed without using poetry remove

[smphhh]

Description

If a local directory dependency has been installed previously in editable mode, but then the directory and the references in pyproject.toml and the lock file are removed via a version control pull, for example, `poetry install --sync` will fail with an error similar to

`Directory /Users/xxx/my_package for my-package does not seem to be a Python package`

Workarounds

The removed dependency can be removed from the venv by running

`poetry run pip uninstall my-package`

After this `poetry install --sync` will work again.

Poetry Installation Method

pipx

Operating System

MacOS 14.3.1

Poetry Version

1.8.1

Poetry Configuration

cache-dir = "/Users/xxx/Library/Caches/pypoetry"
experimental.system-git-client = false
installer.max-workers = null
installer.modern-installation = true
installer.no-binary = null
installer.parallel = true
keyring.enabled = true
solver.lazy-wheel = true
virtualenvs.create = true
virtualenvs.in-project = true
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.no-setuptools = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}/virtualenvs"  # /Users/xxx/Library/Caches/pypoetry/virtualenvs
virtualenvs.prefer-active-python = false
virtualenvs.prompt = "{project_name}-py{python_version}"
warnings.export = true

Python Sysconfig

No response

Example pyproject.toml

No response

Poetry Runtime Logs

Loading configuration file /Users/xxx/poetry.toml
Using virtualenv: /Users/xxx/.venv
Installing dependencies from lock file

Finding the necessary packages for the current system
Path /Users/xxx/my_package for my-package does not exist

  Stack trace:

  10  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/cleo/application.py:327 in run
       325│ 
       326│             try:
     → 327│                 exit_code = self._run(io)
       328│             except BrokenPipeError:
       329│                 # If we are piped to another process, it may close early and send a

   9  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/console/application.py:190 in _run
       188│         self._load_plugins(io)
       189│ 
     → 190│         exit_code: int = super()._run(io)
       191│         return exit_code
       192│ 

   8  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/cleo/application.py:431 in _run
       429│             io.input.interactive(interactive)
       430│ 
     → 431│         exit_code = self._run_command(command, io)
       432│         self._running_command = None
       433│ 

   7  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/cleo/application.py:473 in _run_command
       471│ 
       472│         if error is not None:
     → 473│             raise error
       474│ 
       475│         return terminate_event.exit_code

   6  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/cleo/application.py:457 in _run_command
       455│ 
       456│             if command_event.command_should_run():
     → 457│                 exit_code = command.run(io)
       458│             else:
       459│                 exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED

   5  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/cleo/commands/base_command.py:117 in run
       115│         io.input.validate()
       116│ 
     → 117│         return self.execute(io) or 0
       118│ 
       119│     def merge_application_definition(self, merge_args: bool = True) -> None:

   4  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/cleo/commands/command.py:61 in execute
        59│ 
        60│         try:
     →  61│             return self.handle()
        62│         except KeyboardInterrupt:
        63│             return 1

   3  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/console/commands/install.py:153 in handle
       151│         self.installer.verbose(self.io.is_verbose())
       152│ 
     → 153│         return_code = self.installer.run()
       154│ 
       155│         if return_code != 0:

   2  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/installation/installer.py:104 in run
       102│             self.verbose(True)
       103│ 
     → 104│         return self._do_install()
       105│ 
       106│     def dry_run(self, dry_run: bool = True) -> Installer:

   1  ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/installation/installer.py:337 in _do_install
       335│             if dep.is_file() or dep.is_directory():
       336│                 dep = cast("PathDependency", dep)
     → 337│                 dep.validate(raise_error=not op.skipped)
       338│ 
       339│         # Execute operations

  ValueError

  Path /Users/xxx/my_package for my-package does not exist

  at ~/Library/Application Support/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/core/packages/path_dependency.py:82 in validate
       78│     def validate(self, *, raise_error: bool) -> bool:
       79│         if not self._validation_error:
       80│             return True
       81│         if raise_error:
    →  82│             raise ValueError(self._validation_error)
       83│         logger.warning(self._validation_error)
       84│         return False
       85│ 
       86│     @property

[dimbleby]

probably sensible to update dep.validate(raise_error=not op.skipped) so that it doesn't raise an error for uninstalls (op.job_type == "uninstall") - pull request welcome I expect

[jnhyperion]

I encountered the same bug, here're the steps to reproduce:

1. there's one certain dep example in pyproject.toml
2. poetry install --sync
3. manually install example from source with editable version pip install -e /path/to/example
4. remove dep example in pyproject.toml
5. poetry lock --no-update
6. poetry install --sync

I did some investigations, I think the root cause is poetry use the incorrect distribution of editable installations. when there're editable library in the python env, the editable path, /path/to/example/src in this case, will be always append in the sys.path, and in this code: https://github.com/python-poetry/poetry/blob/main/src/poetry/repositories/installed_repository.py#L241 poetry will always pick example.egg-info from this path, the example.egg-info is generated by the pip install command. while example.egg-info is not the actual distribution, you can use meta.distribution("example") to get the correct one, the correct one should be example-{version}.dist-info.

Unfortunately, I did not manage to find a proper way to fix this without impact the previous logic too much.

[tony]

Getting this as of poetry 1.8.3 on python 3.10.1

[dimbleby]

There is a suggested fix here, if this is important to you then please submit a merge request.