pip3 broken? - Githubissues

selfagency commented 1 year ago

What did you expect to happen?

Python packages should have updated. Running pip3 list --outdated --format=freeze | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 pip3 install -U works without issue.

What actually happened?

―― 09:38:23 - pip3 ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
Requirement already satisfied: pip in ./Library/Python/3.9/lib/python/site-packages (22.2)
Collecting pip
  Using cached pip-22.2.1-py3-none-any.whl (2.0 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 22.2
    Uninstalling pip-22.2:
      Successfully uninstalled pip-22.2
  Rolling back uninstall of pip
  Moving to /Users/daniel/Library/Python/3.9/bin/
   from /Users/daniel/Library/Python/3.9/~in
  Moving to /Users/daniel/Library/Python/3.9/lib/python/site-packages/pip-22.2.dist-info/
   from /Users/daniel/Library/Python/3.9/lib/python/site-packages/~ip-22.2.dist-info
  Moving to /Users/daniel/Library/Python/3.9/lib/python/site-packages/pip/
   from /Users/daniel/Library/Python/3.9/lib/python/site-packages/~ip
ERROR: Exception:
Traceback (most recent call last):
  File "/Users/daniel/Library/Python/3.9/lib/python/site-packages/pip/_internal/cli/base_command.py", line 167, in exc_logging_wrapper
    status = run_func(*args)
  File "/Users/daniel/Library/Python/3.9/lib/python/site-packages/pip/_internal/cli/req_command.py", line 247, in wrapper
    return func(self, options, args)
  File "/Users/daniel/Library/Python/3.9/lib/python/site-packages/pip/_internal/commands/install.py", line 461, in run
    installed = install_given_reqs(
  File "/Users/daniel/Library/Python/3.9/lib/python/site-packages/pip/_internal/req/__init__.py", line 73, in install_given_reqs
    requirement.install(
  File "/Users/daniel/Library/Python/3.9/lib/python/site-packages/pip/_internal/req/req_install.py", line 752, in install
    scheme = get_scheme(
  File "/Users/daniel/Library/Python/3.9/lib/python/site-packages/pip/_internal/locations/__init__.py", line 244, in get_scheme
    from . import _distutils
ImportError: cannot import name '_distutils' from 'pip._internal.locations' (/Users/daniel/Library/Python/3.9/lib/python/site-packages/pip/_internal/locations/__init__.py)

Retry? (y)es/(N)o/(s)hell/(q)uit%

Additional Details

Which operating system or Linux distribution are you using? Mac OS 12.5
How did you install Topgrade? Homebrew
Which version are you running? 9.0.1

This won't help because I ran it after I manually upgraded the packages:

09:40:59 AM ✦2 » topgrade --dry-run --only pip3

―― 09:41:18 - pip3 ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
Dry running: /opt/homebrew/bin/python3 -m pip install --upgrade --user pip

―― 09:41:18 - Summary ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
pip3: OK

Same here:

``` 2022-07-28T13:41:42.419Z DEBUG topgrade::config > Configuration at /Users/daniel/.config/topgrade.toml 2022-07-28T13:41:42.419Z DEBUG topgrade::config > Loaded configuration: ConfigFile { pre_commands: Some({}), post_commands: None, commands: Some({}), git_repos: None, predefined_git_repos: None, disable: None, ignore_failures: None, remote_topgrades: None, remote_topgrade_path: None, ssh_arguments: None, git_arguments: None, tmux_arguments: None, set_title: None, display_time: None, assume_yes: None, yay_arguments: None, no_retry: None, run_in_tmux: None, cleanup: None, notify_each_step: None, accept_all_windows_updates: None, bashit_branch: None, only: None, composer: Some(Composer { self_update: None }), brew: Some(Brew { greedy_cask: None }), linux: Some(Linux { yay_arguments: None, arch_package_manager: None, show_arch_news: None, trizen_arguments: None, pikaur_arguments: None, dnf_arguments: None, apt_arguments: None, enable_tlmgr: None, redhat_distro_sync: None, rpm_ostree: None, emerge_sync_flags: None, emerge_update_flags: None }), git: Some(Git { max_concurrency: None, arguments: None, repos: None, pull_predefined: None }), windows: Some(Windows { accept_all_updates: None, self_rename: None, open_remotes_in_new_terminal: None, enable_winget: None }), npm: Some(NPM { use_sudo: None }), vim: None, firmware: Some(Firmware { upgrade: None }), vagrant: None, flatpak: Some(Flatpak { use_sudo: None }) } 2022-07-28T13:41:42.419Z DEBUG topgrade > Version: 9.0.1 2022-07-28T13:41:42.419Z DEBUG topgrade > OS: aarch64-apple-darwin 2022-07-28T13:41:42.419Z DEBUG topgrade > Args { inner: ["topgrade", "--only", "pip3", "-v"] } 2022-07-28T13:41:42.419Z DEBUG topgrade > Binary path: Ok("/opt/homebrew/bin/topgrade") 2022-07-28T13:41:42.419Z DEBUG topgrade > Self Update: false 2022-07-28T13:41:42.419Z DEBUG topgrade::utils > Detected "/opt/homebrew/bin/git" as "git" 2022-07-28T13:41:42.420Z DEBUG topgrade::utils > Cannot find "doas" 2022-07-28T13:41:42.420Z DEBUG topgrade::utils > Detected "/usr/bin/sudo" as "sudo" 2022-07-28T13:41:42.420Z DEBUG topgrade::utils > Detected "/usr/local/bin/pwsh" as "pwsh" 2022-07-28T13:41:42.730Z TRACE topgrade::executor > Output of "/usr/local/bin/pwsh" "-NoProfile" "-Command" "Split-Path $profile": Output { status: ExitStatus(unix_wait_status(0)), stdout: "/Users/daniel/.config/powershell\n", stderr: "" } 2022-07-28T13:41:42.730Z DEBUG topgrade::utils > Path "/Users/daniel/.config/emacs" doesn't exist 2022-07-28T13:41:42.730Z DEBUG topgrade::utils > Path "/Users/daniel/.emacs.d" doesn't exist 2022-07-28T13:41:42.730Z DEBUG topgrade::steps::git > /Users/daniel/.ideavimrc does not exists 2022-07-28T13:41:42.730Z DEBUG topgrade::steps::git > /Users/daniel/.intellimacs does not exists 2022-07-28T13:41:42.730Z DEBUG topgrade::steps::git > /Users/daniel/.config/.zshrc is a file. Checking /Users/daniel/.config 2022-07-28T13:41:42.730Z DEBUG topgrade::steps::git > Checking if /Users/daniel/.config is a git repository 2022-07-28T13:41:42.734Z TRACE topgrade::executor > Output of "/opt/homebrew/bin/git" "rev-parse" "--show-toplevel": Output { status: ExitStatus(unix_wait_status(32768)), stdout: "", stderr: "fatal: not a git repository (or any of the parent directories): .git\n" } 2022-07-28T13:41:42.734Z DEBUG topgrade::steps::git > Checking if /Users/daniel/.config/fish is a git repository 2022-07-28T13:41:42.737Z TRACE topgrade::executor > Output of "/opt/homebrew/bin/git" "rev-parse" "--show-toplevel": Output { status: ExitStatus(unix_wait_status(32768)), stdout: "", stderr: "fatal: not a git repository (or any of the parent directories): .git\n" } 2022-07-28T13:41:42.737Z DEBUG topgrade::steps::git > /Users/daniel/Library/Application Support/openbox does not exists 2022-07-28T13:41:42.737Z DEBUG topgrade::steps::git > /Users/daniel/Library/Application Support/bspwm does not exists 2022-07-28T13:41:42.737Z DEBUG topgrade::steps::git > /Users/daniel/Library/Application Support/i3 does not exists 2022-07-28T13:41:42.737Z DEBUG topgrade::steps::git > /Users/daniel/Library/Application Support/sway does not exists 2022-07-28T13:41:42.737Z DEBUG topgrade::runner > Step "pip3" 2022-07-28T13:41:42.737Z DEBUG topgrade::utils > Detected "/opt/homebrew/bin/python3" as "python3" 2022-07-28T13:41:42.876Z TRACE topgrade::executor > Output of "/opt/homebrew/bin/python3" "-m" "pip": Output { status: ExitStatus(unix_wait_status(0)), stdout: "\nUsage: \n /opt/homebrew/opt/python@3.9/bin/python3.9 -m pip [options]\n\nCommands:\n install Install packages.\n download Download packages.\n uninstall Uninstall packages.\n freeze Output installed packages in requirements format.\n inspect Inspect the python environment.\n list List installed packages.\n show Show information about installed packages.\n check Verify installed packages have compatible dependencies.\n config Manage local and global configuration.\n search Search PyPI for packages.\n cache Inspect and manage pip's wheel cache.\n index Inspect information available from package indexes.\n wheel Build wheels from your requirements.\n hash Compute hashes of package archives.\n completion A helper command used for command completion.\n debug Show information useful for debugging.\n help Show help for commands.\n\nGeneral Options:\n -h, --help Show help.\n --debug Let unhandled exceptions propagate outside the\n main subroutine, instead of logging them to\n stderr.\n --isolated Run pip in an isolated mode, ignoring\n environment variables and user configuration.\n --require-virtualenv Allow pip to only run in a virtual environment;\n exit with an error otherwise.\n -v, --verbose Give more output. Option is additive, and can be\n used up to 3 times.\n -V, --version Show version and exit.\n -q, --quiet Give less output. Option is additive, and can be\n used up to 3 times (corresponding to WARNING,\n ERROR, and CRITICAL logging levels).\n --log Path to a verbose appending log.\n --no-input Disable prompting for input.\n --proxy Specify a proxy in the form\n scheme://[user:passwd@]proxy.server:port.\n --retries Maximum number of retries each connection should\n attempt (default 5 times).\n --timeout Set the socket timeout (default 15 seconds).\n --exists-action Default action when a path already exists:\n (s)witch, (i)gnore, (w)ipe, (b)ackup, (a)bort.\n --trusted-host Mark this host or host:port pair as trusted,\n even though it does not have valid or any HTTPS.\n --cert Path to PEM-encoded CA certificate bundle. If\n provided, overrides the default. See 'SSL\n Certificate Verification' in pip documentation\n for more information.\n --client-cert Path to SSL client certificate, a single file\n containing the private key and the certificate\n in PEM format.\n --cache-dir Store the cache data in .\n --no-cache-dir Disable the cache.\n --disable-pip-version-check\n Don't periodically check PyPI to determine\n whether a new version of pip is available for\n download. Implied with --no-index.\n --no-color Suppress colored output.\n --no-python-version-warning\n Silence deprecation warnings for upcoming\n unsupported Pythons.\n --use-feature Enable new functionality, that may be backward\n incompatible.\n --use-deprecated Enable deprecated functionality, that will be\n removed in the future.\n", stderr: "" } ―― 09:41:42 - pip3 ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― 2022-07-28T13:41:42.876Z DEBUG topgrade::executor > Running "/opt/homebrew/bin/python3" "-m" "pip" "install" "--upgrade" "--user" "pip" Requirement already satisfied: pip in /opt/homebrew/lib/python3.9/site-packages (22.2.1) ―― 09:41:43 - Summary ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――― pip3: OK 2022-07-28T13:41:43.361Z DEBUG topgrade::terminal > Desktop notification: Topgrade finished successfully ```

MCOfficer commented 1 year ago

Looks like a pip issue: https://github.com/pypa/pip/issues/11319

tos-kamiya commented 1 year ago

I experienced the same error.

I understand that it is a pip error, but if topgrade would update pip first (python3 -m pip install --upgrade pip), it would prevent the users from experiencing this error in the future.

Requirement already satisfied: pip in ./.local/lib/python3.8/site-packages (22.2)
Collecting pip
  Using cached pip-22.2.1-py3-none-any.whl (2.0 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 22.2
    Uninstalling pip-22.2:
      Successfully uninstalled pip-22.2
  Rolling back uninstall of pip
  Moving to /home/toshihiro/.local/bin/pip
   from /tmp/pip-uninstall-37ltymkz/pip
  Moving to /home/toshihiro/.local/bin/pip3
   from /tmp/pip-uninstall-37ltymkz/pip3
  Moving to /home/toshihiro/.local/bin/pip3.8
   from /tmp/pip-uninstall-37ltymkz/pip3.8
  Moving to /home/toshihiro/.local/lib/python3.8/site-packages/pip-22.2.dist-info/
   from /home/toshihiro/.local/lib/python3.8/site-packages/~ip-22.2.dist-info
  Moving to /home/toshihiro/.local/lib/python3.8/site-packages/pip/
   from /home/toshihiro/.local/lib/python3.8/site-packages/~ip
ERROR: Exception:
Traceback (most recent call last):
  File "/home/toshihiro/.local/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 167, in exc_logging_wrapper
    status = run_func(*args)
  File "/home/toshihiro/.local/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 247, in wrapper
    return func(self, options, args)
  File "/home/toshihiro/.local/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 461, in run
    installed = install_given_reqs(
  File "/home/toshihiro/.local/lib/python3.8/site-packages/pip/_internal/req/__init__.py", line 73, in install_given_reqs
    requirement.install(
  File "/home/toshihiro/.local/lib/python3.8/site-packages/pip/_internal/req/req_install.py", line 752, in install
    scheme = get_scheme(
  File "/home/toshihiro/.local/lib/python3.8/site-packages/pip/_internal/locations/__init__.py", line 244, in get_scheme
    from . import _distutils
ImportError: cannot import name '_distutils' from 'pip._internal.locations' (/home/toshihiro/.local/lib/python3.8/site-packages/pip/_internal/locations/__init__.py)

MCOfficer commented 1 year ago

Are you sure? By my understanding, upgrading pip first would just make it fail sooner. I don't think it's within topgrade's scope to account for unexpected behavior of tools like pip.

tos-kamiya commented 1 year ago

Since topgrade is doing carefully rustup before the cargo update, it would be nice to have the same behavior for pip, by updating the pip itself before the pip update. However, I also understand the policy of not addressing it because it is a pip-specific error, as you mentioned.

MCOfficer commented 1 year ago

By my understanding, rustup runs so any following cargo builds don't fail due to an outdated toolchain. If the same can be said about pip (i wouldn't know), that's a valid argument. But this issue is more akin to rustup update failing with a segfault - at which point we run it is not really relevant, it's an upstream issue either way.

tos-kamiya commented 1 year ago

In my experience, there have been several times when the latest version of some library failed to install because the pip was out of date. So, in my mind, if pip fails, the right thing to do is to upgrade pip itself.

However, it seems that the upgrade of pip itself can cause problems in a Windows environment, although I don't use Windows very often. https://github.com/pypa/pip/issues/9395

So, I reconsider that it may be a hasty decision to include the upgrade function of pip itself in topgrade as a default behavior.

joschi commented 1 year ago

Until this has been fixed in pip 22.2.2, you can update pip manually using this command (from https://github.com/pypa/pip/issues/11319#issuecomment-1200119836).

python -m pip install --user --upgrade https://github.com/pypa/pip/archive/refs/tags/22.2.1.zip
# python3 -m pip install --user --upgrade https://github.com/pypa/pip/archive/refs/tags/22.2.1.zip

r-darwish / topgrade

pip3 broken? #983

What did you expect to happen?

What actually happened?

Additional Details