'pip_version_check' in outdated.py does not respect local index-url within a virtualenv

[cooperlees]

• Pip version: pip 9.0.1 from /home/cooper/virtualenvs/wheel_fun/lib/python3.6/site-packages (python 3.6)
• Python version: 3.6
• Operating system: CentOS Linux release 7.3.1611 (Core)

Description:

Running pip commands in a virtualenv where no direct Internet connection is available to hang as it's trying to talk to pypi.python.org. We have a local PyPI Mirror (synced via bandersnatch) running that is reachable. It's configured as such:

• I have not debugged if it happens in a non virtualenv environment.

[global] timeout = 10 index-url = https://pypi.somecompanyiworkfor.com/simple

• Follow ups: 1) Is there another config var to make pip try and update from the local mirror I've missed 2) Would people be fine with me PR'in a diff to allow outdated.py to respect this setting

What I've run:

pip freeze -v: http://pastebin.com/iL2ZtCaQ

[dstufft]

@cooperlees The problem is that the outdated code uses PyPI's JSON API and it sort of purposely talks directly to PyPI and ignores the local repositories so that a stale mirror doesn't keep people pinned back.

So there are a few options here:

• Ditch the idea that we'll work around stale mirrors and just convert this to using the repository API (potentially uses more HTTP requests).
• More aggressively time out the connection so it hangs for a shorter amount of time.
• Do the HTTP requests at the very start of the pip process, but spun off in a dedicated thread so that it's non-blocking, and if the HTTP request hasn't completed by the time we're ready to exit just bail out on trying to do the check.
• Decide this is working as intended and folks like you will just need to use pip --disable-pip-version-check ....

[cooperlees]

Thanks for the quick reply! 1) I feel if people allow their local mirror to go stale that's their own problem 🤡 ... #monitoring

• I like the idea of ditching the JSON API or adding a config param to let people choose to do so
• I could see if the timeout option in pip.conf times out harder - at the moment i have it sent to 10 - but this is less desirable
• Wish we could asyncio the http requests, dam Python 2 people :P - But kind of just hiding the issue
• @carljm and I talked about disabling the version check, but I would rather hit our local mirror and update if there is an update ... If it has to come to this I'd like to do it via pip.conf as well and not the cli (so I can automate turning it off for everyone who creates a virtualenv here @ FB)

How about for now I look at a 'use_local_mirror_for_pip_upgrade' option in pip.conf and adding code to respect that for those who want it in outdated.py + deps?

[dstufft]

Yea, it can be done via configuration too (all of pip's options are available as a CLI flag, an environment variable, and configuration flag).

Overall though, I am generally against adding configuration options if we can help it. I'd personally prefer to just drop the use of JSON all together and rely on PackageFinder configured the same as pip install would to locate the latest version rather than adding a configuration option to switch between them. How the outdated check happens should be an implementation detail.

If this seems like something that would work for you, the quickest way to get this change in would be to submit a PR.

[cooperlees]

Working on a PR: https://github.com/pypa/pip/pull/4308

[vphilippon]

I think this has fallen in a crack and was fixed by #4336

[cooperlees]

Yes. That was me. Been waiting for 9.1.0 to confirm, but let's just assume it works for now.