--no-index not recognising whl

james-smith-uk commented 7 months ago

Description

Pip appears to ignores the --no-links command for some wheels hosted on GitHub. Instead pip attempts to retrieve the wheel again from github.

Expected behavior

For other libraries sourced from github, no outbound connection is made - for example: flash-attn @ https://github.com/Dao-AILab/flash-attention/releases/download/v2.3.3/flash_attn-2.3.3+cu116torch2.0cxx11abiFALSE-cp310-cp310-linux_x86_64.whl

pip version

22.3.1

Python version

3.10

OS

Linux

How to Reproduce

Downloaded the following whl : https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.7.1/en_core_web_sm-3.7.1-py3-none-any.whl
Add the following to the requirements.txt file en_core_web_sm @ https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-3.7.1/en_core_web_sm-3.7.1-py3-none-any.whl
Run the following comman: pip install --upgrade -r requirements.txt --no-index --find-links python-libs/

Output

No response

Code of Conduct

[X] I agree to follow the PSF Code of Conduct.

notatallshaw commented 6 months ago

For other libraries sourced from github, no outbound connection is made - for example: flash-attn @ https://github.com/Dao-AILab/flash-attention/releases/download/v2.3.3/flash_attn-2.3.3+cu116torch2.0cxx11abiFALSE-cp310-cp310-linux_x86_64.whl

I don't think this is correct, when I run this requirement on a platform that matches that matches the wheel it does download it:

$ pip install --upgrade --dry-run -r requirements.txt --no-index --find-links wheels/
Looking in links: wheels/
Collecting flash-attn@ https://github.com/Dao-AILab/flash-attention/releases/download/v2.3.3/flash_attn-2.3.3+cu116torch2.0cxx11abiFALSE-cp310-cp310-linux_x86_64.whl
  Downloading https://github.com/Dao-AILab/flash-attention/releases/download/v2.3.3/flash_attn-2.3.3+cu116torch2.0cxx11abiFALSE-cp310-cp310-linux_x86_64.whl (30.2 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 30.2/30.2 MB 31.4 MB/s eta 0:00:00
ERROR: Could not find a version that satisfies the requirement einops (from flash-attn) (from versions: none)
ERROR: No matching distribution found for einops

I only see no attempt to download it when I'm running on a platform that doesn't match the wheel:

$ pip install --upgrade --dry-run -r requirements.txt --no-index --find-links wheels/
Looking in links: wheels/
ERROR: flash_attn-2.3.3+cu116torch2.0cxx11abiFALSE-cp310-cp310-linux_x86_64.whl is not a supported wheel on this platform

The error here though is clearly different it says "is not a supported wheel on this platform". What error are you seeing when pip does not download a wheel?

I don't know the intention of --no-index when you provide a direct link for the requirement, but it is not looking up an index, and if you provide a direct link I would expect pip to use that, and not a wheel found with --find-links that does not match the url of the direct requirement.

notatallshaw commented 4 months ago

With no further update I am closing this issue as there is nothing further to do. Please feel free to repost the issue if you find steps where pip is not behaving correctly.

pypa / pip