topgrade-rs / topgrade

Upgrade all the things
GNU General Public License v3.0
2.08k stars 141 forks source link

`poetry self update` fails with externally-managed-environment errors #979

Open AThePeanut4 opened 3 weeks ago

AThePeanut4 commented 3 weeks ago

Erroneous Behavior

Topgrade is attempting to run poetry self update, which fails with externally-managed-environment errors because I've installed poetry through pacman.

Expected Behavior

poetry self update should be skipped when poetry is installed into an externally managed environment, similar to the way the pip self-upgrade step is skipped. I believe poetry self update probably also shouldn't be run if poetry is installed through pipx - the docs imply that poetry should be upgraded using pipx upgrade poetry in that case, which would be handled by the pipx step in topgrade.

Steps to reproduce

topgrade --only poetry

Problem persists without calling from topgrade

N/A

Did you run topgrade through Remote Execution

Configuration file (Optional)

N/A

Additional Details

Verbose Output (topgrade -v)

DEBUG Current system locale is en-ZA
DEBUG Configuration at /home/andre/.config/topgrade.toml
DEBUG Loaded configuration: ConfigFile { include: Some(Include { paths: None }), misc: Some(Misc { pre_sudo: None, sudo_command: None, disable: None, ignore_failures: None, remote_topgrades: None, remote_topgrade_path: None, ssh_arguments: None, tmux_arguments: None, set_title: None, display_time: None, assume_yes: None, no_retry: None, run_in_tmux: None, tmux_session_mode: None, cleanup: None, notify_each_step: None, skip_notify: None, bashit_branch: None, only: None, no_self_update: None, log_filters: None }), pre_commands: Some({}), post_commands: Some({}), commands: Some({}), python: Some(Python { enable_pip_review: None, enable_pip_review_local: None, enable_pipupgrade: None, pipupgrade_arguments: None }), composer: Some(Composer { self_update: None }), brew: Some(Brew { greedy_cask: None, greedy_latest: None, greedy_auto_updates: None, autoremove: None, fetch_head: None }), linux: Some(Linux { yay_arguments: None, aura_aur_arguments: None, aura_pacman_arguments: None, arch_package_manager: None, show_arch_news: None, garuda_update_arguments: None, trizen_arguments: None, pikaur_arguments: None, pamac_arguments: None, dnf_arguments: None, nix_arguments: None, nix_env_arguments: None, apt_arguments: None, enable_tlmgr: None, redhat_distro_sync: None, suse_dup: None, rpm_ostree: None, emerge_sync_flags: None, emerge_update_flags: None, home_manager_arguments: None }), git: Some(Git { max_concurrency: None, arguments: None, repos: None, pull_predefined: None }), containers: Some(Containers { ignored_containers: None, runtime: None }), windows: Some(Windows { accept_all_updates: None, self_rename: None, open_remotes_in_new_terminal: None, wsl_update_pre_release: None, wsl_update_use_web_download: None }), npm: Some(NPM { use_sudo: None }), yarn: Some(Yarn { use_sudo: None }), vim: Some(Vim { force_plug_update: None }), firmware: Some(Firmware { upgrade: None }), vagrant: Some(Vagrant { directories: None, power_on: None, always_suspend: None }), flatpak: Some(Flatpak { use_sudo: None }), distrobox: Some(Distrobox { use_root: None, containers: None }), lensfun: Some(Lensfun { use_sudo: None }) }
DEBUG Version: 16.0.1
DEBUG OS: x86_64-unknown-linux-gnu
DEBUG Args { inner: ["topgrade", "-v", "--only", "poetry"] }
DEBUG Binary path: Ok("/usr/bin/topgrade")
DEBUG self-update Feature Enabled: false
DEBUG Configuration: Config { opt: CommandLineArgs { edit_config: false, show_config_reference: false, run_in_tmux: false, cleanup: false, dry_run: false, no_retry: false, disable: [], only: [Poetry], custom_commands: [], env: [], verbose: true, keep_at_end: false, skip_notify: false, yes: None, disable_predefined_git_repos: false, config: None, remote_host_limit: None, show_skipped: false, log_filter: "warn", gen_completion: None, gen_manpage: false, no_self_update: false }, config_file: ConfigFile { include: Some(Include { paths: None }), misc: Some(Misc { pre_sudo: None, sudo_command: None, disable: None, ignore_failures: None, remote_topgrades: None, remote_topgrade_path: None, ssh_arguments: None, tmux_arguments: None, set_title: None, display_time: None, assume_yes: None, no_retry: None, run_in_tmux: None, tmux_session_mode: None, cleanup: None, notify_each_step: None, skip_notify: None, bashit_branch: None, only: None, no_self_update: None, log_filters: None }), pre_commands: Some({}), post_commands: Some({}), commands: Some({}), python: Some(Python { enable_pip_review: None, enable_pip_review_local: None, enable_pipupgrade: None, pipupgrade_arguments: None }), composer: Some(Composer { self_update: None }), brew: Some(Brew { greedy_cask: None, greedy_latest: None, greedy_auto_updates: None, autoremove: None, fetch_head: None }), linux: Some(Linux { yay_arguments: None, aura_aur_arguments: None, aura_pacman_arguments: None, arch_package_manager: None, show_arch_news: None, garuda_update_arguments: None, trizen_arguments: None, pikaur_arguments: None, pamac_arguments: None, dnf_arguments: None, nix_arguments: None, nix_env_arguments: None, apt_arguments: None, enable_tlmgr: None, redhat_distro_sync: None, suse_dup: None, rpm_ostree: None, emerge_sync_flags: None, emerge_update_flags: None, home_manager_arguments: None }), git: Some(Git { max_concurrency: None, arguments: None, repos: None, pull_predefined: None }), containers: Some(Containers { ignored_containers: None, runtime: None }), windows: Some(Windows { accept_all_updates: None, self_rename: None, open_remotes_in_new_terminal: None, wsl_update_pre_release: None, wsl_update_use_web_download: None }), npm: Some(NPM { use_sudo: None }), yarn: Some(Yarn { use_sudo: None }), vim: Some(Vim { force_plug_update: None }), firmware: Some(Firmware { upgrade: None }), vagrant: Some(Vagrant { directories: None, power_on: None, always_suspend: None }), flatpak: Some(Flatpak { use_sudo: None }), distrobox: Some(Distrobox { use_root: None, containers: None }), lensfun: Some(Lensfun { use_sudo: None }) }, allowed_steps: [Poetry] }
DEBUG Detected "/usr/bin/pwsh" as "pwsh"
DEBUG Executing command `/usr/bin/pwsh -NoProfile -Command 'Split-Path $profile'`
DEBUG Path "/home/andre/.config/powershell" exists
DEBUG Path "/home/andre/.config/emacs" doesn't exist
DEBUG Path "/home/andre/.emacs.d" exists
DEBUG Path "/home/andre/.emacs.d/bin/doom" doesn't exist
DEBUG Cannot find "doas"
DEBUG Detected "/usr/bin/sudo" as "sudo"
DEBUG Step "Poetry"
DEBUG Detected "/usr/bin/poetry" as "poetry"

── 15:23:22 - Poetry ───────────────────────────────────────────────────────────
DEBUG Executing command `/usr/bin/poetry self update`
Updating Poetry version ...

Using version ^1.8.4 for poetry

Updating dependencies
Resolving dependencies... (0.1s)

Package operations: 0 installs, 13 updates, 0 removals

  - Updating cryptography (42.0.8 -> 43.0.3): Installing...
  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
  - Updating cryptography (42.0.8 -> 43.0.3): Failed
  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run

  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
  CalledProcessError
  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run

  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'cryptography', '-y']' returned non-zero exit status 1.
  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run

  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
  at /usr/lib/python3.12/subprocess.py:571 in run
  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
       567│             # We don't call process.wait() as .__exit__ does that for us.
  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
       568│             raise
  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
       569│         retcode = process.poll()
  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
       570│         if check and retcode:
  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
    →  571│             raise CalledProcessError(retcode, process.args,
  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
       572│                                      output=stdout, stderr=stderr)
  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
       573│     return CompletedProcess(process.args, retcode, stdout, stderr)
  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
       574│
  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
       575│
  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run

  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
The following error occurred when trying to handle this error:
  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run

  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run

  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
  EnvCommandError
  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run

  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
  Command ['/usr/bin/python', '-m', 'pip', 'uninstall', 'cryptography', '-y'] errored with the following return code 1

  Output:
  error: externally-managed-environment

  × This environment is externally managed
  ╰─> To install Python packages system-wide, try 'pacman -S
      python-xyz', where xyz is the package you are trying to
      install.

      If you wish to install a non-Arch-packaged Python package,
      create a virtual environment using 'python -m venv path/to/venv'.
      Then use path/to/venv/bin/python and path/to/venv/bin/pip.

      If you wish to install a non-Arch packaged Python application,
      it may be easiest to use 'pipx install xyz', which will manage a
      virtual environment for you. Make sure you have python-pipx
      installed via pacman.

  note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
  hint: See PEP 668 for the detailed specification.

  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run

  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
  at /usr/lib/python3.12/site-packages/poetry/utils/env/base_env.py:342 in _run
  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
      338│                 output = subprocess.check_output(
  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
      339│                     cmd, stderr=stderr, env=env, text=True, **kwargs
  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
      340│                 )
  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
      341│         except CalledProcessError as e:
  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
    → 342│             raise EnvCommandError(e)
  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
      343│
  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
      344│         return output
  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
      345│
  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
      346│     def execute(self, bin: str, *args: str, **kwargs: Any) -> int:
  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run

  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
Cannot install cryptography.
  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run

  - Updating more-itertools (10.3.0 -> 10.5.0): Installing...
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
  - Updating more-itertools (10.3.0 -> 10.5.0): Failed
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run

  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
  CalledProcessError
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run

  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'more-itertools', '-y']' returned non-zero exit status 1.
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run

  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
  at /usr/lib/python3.12/subprocess.py:571 in run
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
       567│             # We don't call process.wait() as .__exit__ does that for us.
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
       568│             raise
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
       569│         retcode = process.poll()
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
       570│         if check and retcode:
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
    →  571│             raise CalledProcessError(retcode, process.args,
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
       572│                                      output=stdout, stderr=stderr)
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
       573│     return CompletedProcess(process.args, retcode, stdout, stderr)
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
       574│
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
       575│
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run

  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
The following error occurred when trying to handle this error:
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run

  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run

  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
  EnvCommandError
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run

  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
  Command ['/usr/bin/python', '-m', 'pip', 'uninstall', 'more-itertools', '-y'] errored with the following return code 1

  Output:
  error: externally-managed-environment

  × This environment is externally managed
  ╰─> To install Python packages system-wide, try 'pacman -S
      python-xyz', where xyz is the package you are trying to
      install.

      If you wish to install a non-Arch-packaged Python package,
      create a virtual environment using 'python -m venv path/to/venv'.
      Then use path/to/venv/bin/python and path/to/venv/bin/pip.

      If you wish to install a non-Arch packaged Python application,
      it may be easiest to use 'pipx install xyz', which will manage a
      virtual environment for you. Make sure you have python-pipx
      installed via pacman.

  note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
  hint: See PEP 668 for the detailed specification.

  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run

  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
  at /usr/lib/python3.12/site-packages/poetry/utils/env/base_env.py:342 in _run
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
      338│                 output = subprocess.check_output(
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
      339│                     cmd, stderr=stderr, env=env, text=True, **kwargs
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
      340│                 )
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
      341│         except CalledProcessError as e:
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
    → 342│             raise EnvCommandError(e)
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
      343│
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
      344│         return output
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
      345│
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
      346│     def execute(self, bin: str, *args: str, **kwargs: Any) -> int:
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run

  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
Cannot install more-itertools.
  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run

  - Updating urllib3 (1.26.20 -> 2.2.3): Failed

  CalledProcessError

  Command '['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y']' returned non-zero exit status 1.

  at /usr/lib/python3.12/subprocess.py:571 in run
       567│             # We don't call process.wait() as .__exit__ does that for us.
       568│             raise
       569│         retcode = process.poll()
       570│         if check and retcode:
    →  571│             raise CalledProcessError(retcode, process.args,
       572│                                      output=stdout, stderr=stderr)
       573│     return CompletedProcess(process.args, retcode, stdout, stderr)
       574│
       575│

The following error occurred when trying to handle this error:

  EnvCommandError

  Command ['/usr/bin/python', '-m', 'pip', 'uninstall', 'urllib3', '-y'] errored with the following return code 1

  Output:
  error: externally-managed-environment

  × This environment is externally managed
  ╰─> To install Python packages system-wide, try 'pacman -S
      python-xyz', where xyz is the package you are trying to
      install.

      If you wish to install a non-Arch-packaged Python package,
      create a virtual environment using 'python -m venv path/to/venv'.
      Then use path/to/venv/bin/python and path/to/venv/bin/pip.

      If you wish to install a non-Arch packaged Python application,
      it may be easiest to use 'pipx install xyz', which will manage a
      virtual environment for you. Make sure you have python-pipx
      installed via pacman.

  note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
  hint: See PEP 668 for the detailed specification.

  at /usr/lib/python3.12/site-packages/poetry/utils/env/base_env.py:342 in _run
      338│                 output = subprocess.check_output(
      339│                     cmd, stderr=stderr, env=env, text=True, **kwargs
      340│                 )
      341│         except CalledProcessError as e:
    → 342│             raise EnvCommandError(e)
      343│
      344│         return output
      345│
      346│     def execute(self, bin: str, *args: str, **kwargs: Any) -> int:

Cannot install urllib3.

DEBUG Command failed: Err(
   0: Command failed: `/usr/bin/poetry self update`
   1: `/usr/bin/poetry` failed: exit status: 1

Location:
   src/steps/generic.rs:1028)
DEBUG Step "Poetry" failed:
   0: Command failed: `/usr/bin/poetry self update`
   1: `/usr/bin/poetry` failed: exit status: 1

Location:
   src/steps/generic.rs:1028
Poetry failed:
   0: Command failed: `/usr/bin/poetry self update`
   1: `/usr/bin/poetry` failed: exit status: 1

Location:
   src/steps/generic.rs:1028
Retry? (y)es/(N)o/(s)hell/(q)uit

── 15:23:25 - Summary ──────────────────────────────────────────────────────────
Poetry: FAILED
DEBUG Desktop notification: Topgrade finished with errors
SteveLauC commented 3 weeks ago

Right, that makes sense. Is there anyway we can know if it is externally managed?

AThePeanut4 commented 3 weeks ago

I don't think there's a super straightforward way, but I think this should do the trick (at least on Linux/macOS, not sure how things work on Windows):

  1. Find the poetry wrapper script on PATH, and parse out the Python interpreter it uses from the shebang line. This is simply #!/usr/bin/python for me, and will be something like #!/home/user/.local/share/pipx/venvs/poetry/bin/python if installed using pipx.
  2. Run this script using that interpreter: import sys; import sysconfig; from os import path; from poetry import poetry; stdlib = sysconfig.get_path('stdlib'); print('Y') if (poetry.__file__.startswith(stdlib) and path.isfile(path.join(stdlib, 'EXTERNALLY-MANAGED'))) or path.isfile(path.join(sys.prefix, 'pipx_metadata.json')) else print('N') (modified version of the script used in the pip step). If poetry is installed under the stdlib path and that path is externally managed, or if it is installed a pipx venv, then it will print Y, and N otherwise. pipx adds a pipx_metadata.json file to the venv so I'm using that.

It's a bit convoluted, but it seems to work with me just running the script manually on installations through pacman, pipx, and the official install script.

AThePeanut4 commented 3 weeks ago

Didn't see that the official poetry installer adds a poetry_env file to its venv, so checking for that is probably the easiest solution - this much simpler script does that: import sys; print('Y') if path.isfile(path.join(sys.prefix, 'poetry_env')) else print('N'). The self-update would then only be run if the official installer was used, so it might be useful to add a config option of some sort to bypass the check for those using some kind of exotic setup but still want it to self-update.

SteveLauC commented 3 weeks ago

This approach looks viable.

so it might be useful to add a config option of some sort to bypass the check for those using some kind of exotic setup but still want it to self-update.

You mean the users who want to update poetry with external tools? One can do this by disabling the poetry step.

SteveLauC commented 3 weeks ago

And, just to confirm, that solution would also work on Windows, right?

AThePeanut4 commented 3 weeks ago

You mean the users who want to update poetry with external tools? One can do this by disabling the poetry step.

I mean those that didn't install poetry through the official installer but still want topgrade to run poetry self update - perhaps they installed manually. Effectively an override to skip the check and revert to topgrade's current behaviour of running poetry self update unconditionally.

And, just to confirm, that solution would also work on Windows, right?

I don't think so, because Windows won't have the shebang line which gives the path to the python interpreter that the script needs to be run with. Windows also doesn't have "text" scripts you can run directly, it would have to be some kind of (binary) exe. The actual check for the poetry_env file I imagine should be fine. I'll boot into windows later and try figure something out.

SteveLauC commented 3 weeks ago

I mean those that didn't install poetry through the official installer but still want topgrade to run poetry self update - perhaps they installed manually. Effectively an override to skip the check and revert to topgrade's current behaviour of running poetry self update unconditionally.

Thanks for the explanation, I wonder can do poetry self update to installations done by the official script and the manual one, will this command break manual installations?

I don't think so, because Windows won't have the shebang line which gives the path to the python interpreter that the script needs to be run with. Windows also doesn't have "text" scripts you can run directly, it would have to be some kind of (binary) exe. The actual check for the poetry_env file I imagine should be fine. I'll boot into windows later and try figure something out.

Thanks, appreciate the work.

AThePeanut4 commented 4 days ago

Apologies for the delay, I had figured it out a while ago, but got busy and didn't get around to replying. I was going to explain the process here, but I decided to just go ahead and actually implement it in #989, it wasn't particularly difficult.

Thanks for the explanation, I wonder can do poetry self update to installations done by the official script and the manual one, will this command break manual installations?

It shouldn't break manual installations no - it's running the equivalent of pip install --upgrade inside the poetry venv. A manual installation is basically the same as an official install, installing manually is "essentially performing the steps carried out by the official installer" (from here).