search

pypa / pip

The Python package installer
https://pip.pypa.io/
MIT License
9.52k stars 3.03k forks source link

Optional automatic upgrade of pip on new releases #11591

Closed marcelm closed 1 year ago

marcelm commented 2 years ago

What's the problem this feature will solve?

Whenever there is a new pip release, I get a message like this the next time I run pip install in one of my virtualenvs:

[notice] A new release of pip available: 22.3 -> 22.3.1
[notice] To update, run: pip install --upgrade pip

Because I have never suffered any negative consequences from upgrading (thank you for that) and also because I don’t want to see the message again, I nearly always run the upgrade immediately. However, it does not feel very productive to do this manually.

Describe the solution you'd like

Instead of telling me how to upgrade pip, pip could just do the upgrade.

Here is some brainstorming how this could work. Instead of --disable-pip-version-check, there could be a new option, something like --pip-version-upgraded-action=, with different actions:

Maybe this is going too far, but I could even imagine that ask could become the default 1) if the prompt is only shown when input is a terminal and 2) if the options are not only "yes" and "no", but also "no, and don’t ask again for this virtual environment".

Alternative Solutions

Perhaps an argument could be made that no upgrade notice should be shown by default. They are not shown for other packages in the virtual environment, so why single out pip?

I am aware of --disable-pip-version-check, but as long as consensus is that pip is important enough to be upgraded as soon as a new release comes out (hence notices are shown), I would prefer to do the upgrade.

Code of Conduct

notatallshaw commented 1 year ago

upgrade does the upgrade without prompting

What sub-commands would this trigger on?

Would it happen before the sub-command and then the sub-command is run?

If the user was using the install sub-command would the options the user provided to the install sub-command be passed to the pip upgrade? Such as what index is being used? Or if a --user flag is passed?

marcelm commented 1 year ago

What sub-commands would this trigger on?

Since a decision has already been made when to show notices, I’d just go with that. That is, whichever command currently triggers a notice would trigger an update instead (these seem to be install, download, list, index, and wheel). But in at least my case, just install would be good enough.

Would it happen before the sub-command and then the sub-command is run?

I guess that it does not really matter in practice, but it would feel cleaner to do the upgrade first, at least in the case of the install subcommand.

If the user was using the install sub-command would the options the user provided to the install sub-command be passed to the pip upgrade? Such as what index is being used?

When I use install with --index-url, the upgrade notice doesn’t include --index-url. I don’t know how much sense this makes, but my point is that the notice already shows the recommended command. If the command isn’t correct, then it’s a problem already now.

Or if a --user flag is passed?

Are upgrade notices even triggered with --user? I don’t know whether it’s due to Debian/Ubuntu-specific patches, but using my system pip on Ubuntu, there’s no notice with --user.

uranusjr commented 1 year ago

I would only trigger this on install, and maybe uninstall. Having other seemingly read-only commands silently modifying the environment feels very wrong.

notatallshaw commented 1 year ago

install with --index-url, the upgrade notice doesn’t include --index-url. I don’t know how much sense this makes, but my point is that the notice already shows the recommended command. If the command isn’t correct, then it’s a problem already now.

The command it gives is a best suggestion on how to upgrade, for example on Windows pip can not guarantee the quoting of the executable so it doesn't attempt to quote it and in many situations copying and pasting it will fail.

I used to work in a large company where people regularly passed in --index-url to get Pip to point a private proxy of Pypi, if the command did not pass in --index-url it would fail to download anything. So what should pip do with the rest of the install if the upgrade pip step fails?

Are upgrade notices even triggered with --user? I don’t know whether it’s due to Debian/Ubuntu-specific patches, but using my system pip on Ubuntu, there’s no notice with --user.

Seems to give the notice for me:

$ python3 -m pip install attrs --user
Requirement already satisfied: attrs in /usr/lib/python3/dist-packages (21.2.0)
WARNING: You are using pip version 22.0.4; however, version 22.3.1 is available.
You should consider upgrading via the '/usr/bin/python3 -m pip install --upgrade pip' command.

btw I don't have any strong opinions on how this should be implemented, just immediate clarifying questions that come to mind.

cchadha2 commented 1 year ago

In terms of the upgrade option, would you print the notice and then execute the command? At the very least this would provide some context in logs.

Having said that, I'm generally not a fan of implicitly upgrading a tool as the user executes commands. Constraining this to install and uninstall might make sense but introduces inconsistency between when the notice is printed and when the implicit upgrade is performed.

I wonder if it'd be useful to check how npm handles this...AFAIK the upgrade step is always explicit?

pfmoore commented 1 year ago

If a Windows user installs a package xxx using pip install xxx, then the pip.exe wrapper is in use, and cannot be upgraded. What would the automatic upgrade do in that case?

Having said this, I’m basically -1 on the idea. I’m not aware of any other program that upgrades itself without the user invoking a dedicated “self upgrade” command, and I think that doing so would be a source of difficult to debug issues.

marcelm commented 1 year ago

Trying to address the above comments in one go, here’s an updated suggestion.

I wonder if it'd be useful to check how npm handles this...AFAIK the upgrade step is always explicit?

I don’t have experience with npm, but from what I can tell playing around with it for 15 minutes, it appears that npm doesn’t show upgrade notices on every install as pip does.

notatallshaw commented 1 year ago

I’m not aware of any other program that upgrades itself without the user invoking a dedicated “self upgrade” command, and I think that doing so would be a source of difficult to debug issues.

FYI Conda does it by default! https://conda.io/projects/conda/en/latest/user-guide/configuration/use-condarc.html#auto-update-conda

The behavior is that you will be on the new version of conda the next time you run a conda command, the current conda command will run on the old version.

The main issue in my experience (and this is more to do with it being default) is it exposes a lot of users to unexpected bugs making it become extremely time critical to roll out fixes following major releases.

marcelm commented 1 year ago

If a Windows user installs a package xxx using pip install xxx, then the pip.exe wrapper is in use, and cannot be upgraded. What would the automatic upgrade do in that case?

I don’t use Windows that much and don’t know what "cannot be upgraded means". Is this about pip install vs. python -m pip install? Then I assume that this situation can be detected, and an upgrade wouldn’t be attempted. That’s fine IMO – it doesn’t need to work in all cases.

Having said this, I’m basically -1 on the idea. I’m not aware of any other program that upgrades itself without the user invoking a dedicated “self upgrade” command, and I think that doing so would be a source of difficult to debug issues.

Thanks @notatallshaw for the Conda example, I use Conda regularly and wasn’t even aware it auto-updates.

Another example for auto-upgrades is Debian/Ubuntu’s "unattended upgrades" mechanism, and something like this can be enabled for other Linux distributions as well. If I remember correctly, Windows also updates itself quite regularly, even without any user interaction.

Running non-interactive auto-upgrades by default is not a good idea, I agree. I think there should at least be a prompt. Then possibly, for those who want, the prompt could be configured away, giving them non-interactive autoupgrades like Conda.

I assume you mostly meant "command-line programs somewhat like pip" and I agree it’s unusual for a command-line program to update itself, but then on the other hand, it’s also unusual to print upgrade notices.

That is probably my main gripe: By printing the notice, pip gives the impression that it is important to update, but then it just prints the command, not making it as easy as it could be. It feels weird to get me as a human involved in running a command that pip could just as well run itself. I realize now it’s not quite as easy as that in all cases, but I assume it could be made to work in many cases.

Finally, I should mention that removing the update notices altogether would also solve the problem for me ...

pfmoore commented 1 year ago

Finally, I should mention that removing the update notices altogether would also solve the problem for me ...

Well, the --disable-pip-version-check option (which can be set in the config file) does that, but you said originally that wasn't a suitable solution for you.

pip gives the impression that it is important to update

It's important to upgrade for two main reasons:

  1. We don't support any versions of pip other than the latest, so if you don't upgrade regularly, you'll be unsupported. That may be fine for you, of course.
  2. If you don't use an up to date version of pip, you won't get the benefit of new standards that get developed. For example, you could get known-problematic packages installed if your pip doesn't support the "yanked" flag on PyPI. Or packages that are not compatible with your Python if you have a version of pip without support for the "requires_python" metadata. Too many people using out of date versions of pip would significantly increase the risk of breakage when we deploy new standards like this.

You can, of course, make your own choice around upgrading. But the upgrade notice does reflect our advice on whether you should or not.

The reasons we don't automatically upgrade, but just let the user know, are basically:

  1. It's impolite to upgrade someone's software without giving them a choice. OS-level upgrades are a different matter, and there's typically a lot of infrastructure to allow people (typically IT departments) to customise the process. A command line tool like pip doesn't have that infrastructure, and people don't expect automatic updates.
  2. It's technically extremely tricky to do right (or even to detect when it's possible to do it safely). A failed automatic update is the worst possible outcome, as you've now just broken the user's ability to fix the issue as well.
  3. People should be managing the upgrades. Even though we say "there's a new version, you should upgrade" that doesn't mean that we expect people to immediately deploy the new pip to production[^1]. Testing the new release and deploying it once you're sure it works for you should be how you respond to that notification, and no automatic update can do that for you.

[^1]: Even though sadly, a lot of people do 🙁

potiuk commented 1 year ago

My comment: All that @pfmoore writes is very accurate. Trying to upgrade automatically - especially 'self-upgrade' is a recipe for disaster. There are many scenarios that can go wrong.

Also as a general reasoning, it is generally a bad idea from security point of view. User should never be surprised by a new software coming from remote when they do not expect it to happen. In a number of scenarios it's even enforced (permissions of the user, requiring sudo, selinux and a number of others) completely outside of the realm and capabilties of pip command that is being executed and sandbox it runs in and user it runs with.

pradyunsg commented 1 year ago

--disable-pip-version-check

And, as with every CLI option in pip, this can also be configured in pip's configuration files. See https://pip.pypa.io/en/stable/topics/configuration/#naming for the naming convention and how that works.

marcelm commented 1 year ago

Thanks all for your time engaging with me. I don’t think I have managed particularly well to convey that I don’t want to force an auto-update on anyone – my favorite option would be to prompt, and then it would be fully under the user’s control. However, it also appears to be more complicated than I thought (as always) and enthusiasm seems to be pretty low for the idea in general. I would not be able to submit a PR for this myself anyway, so I will let this go. It’s good to have your stance on the idea documented here. (Feel free to close.)

pradyunsg commented 1 year ago

Thanks for this discussion folks! :)