Do not stop yielding links matching filter on first match

EvaSDK commented 1 year ago

Following the release of version 2.0, I was checking if everything was working as intended. While devpi list filtering works as documented in the README, actual usage with pip still does not filter according to constraints correctly.

To test this, I created the constrained index following documentation, then added the following constraint:

$ devpi index -c root/devpi type=constrained bases=root/pypi constraints="tox~=4.0.0"
http://localhost:3141/root/devpi?no_projects=:
  type=constrained
  bases=root/pypi
  volatile=True
  acl_upload=root
  acl_toxresult_upload=:ANONYMOUS:
  constraints=tox~=4.0.0
  mirror_whitelist=
  mirror_whitelist_inheritance=intersection

devpi list works as expected:

$ devpi list --all tox
http://localhost:3141/root/pypi/+f/2a5/1d4fd98a93b24/tox-4.0.19-py3-none-any.whl
http://localhost:3141/root/pypi/+f/31d/95663dc66f8d5/tox-4.0.19.tar.gz
http://localhost:3141/root/pypi/+f/2e3/fa6d1b5cc09a9/tox-4.0.18-py3-none-any.whl
http://localhost:3141/root/pypi/+f/e86/2abf5b75aa2d8/tox-4.0.18.tar.gz
[...]
http://localhost:3141/root/pypi/+f/6aa/558b74bb2d96c/tox-4.0.1-py3-none-any.whl
http://localhost:3141/root/pypi/+f/c7c/d95e5e567a7d1/tox-4.0.1.tar.gz
http://localhost:3141/root/pypi/+f/e6a/dcebddfec5e45/tox-4.0.0-py3-none-any.whl
http://localhost:3141/root/pypi/+f/176/43b0e29c41f78/tox-4.0.0.tar.gz

but running pip install returned the following unexpected result:

$ pip download --no-deps --index-url=http://localhost:3141/root/devpi tox ; rm tox*.whl
Looking in indexes: http://localhost:3141/root/devpi
Collecting tox
  Downloading http://localhost:3141/root/pypi/%2Bf/e3d/4a65852f029e5/tox-4.4.6-py3-none-any.whl (148 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 149.0/149.0 kB 475.8 MB/s eta 0:00:00
Saved ./tox-4.4.6-py3-none-any.whl
Successfully downloaded tox

Removing the break fixes the issue as it allows ConstrainedStage to filter all versions.

$ pip download --no-deps --index-url=http://localhost:3141/root/devpi tox ; rm tox*.whl
Looking in indexes: http://localhost:3141/root/devpi
Collecting tox
  Downloading http://localhost:3141/root/pypi/%2Bf/2a5/1d4fd98a93b24/tox-4.0.19-py3-none-any.whl (144 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 144.0/144.0 kB 453.8 MB/s eta 0:00:00
Saved ./tox-4.0.19-py3-none-any.whl
Successfully downloaded tox

I think the issue came from my PR keeping this break while it was not necessary with SimpleLinkMeta since it was addressed at the loop trying to find the proper version split in the tuple-based link info.

I could not find a way to quickly add a test case in the test suite as I am not familiar with Pyramid but if you point me at which function I can use to see the links the same way pip does, I can work on adding it.

fschulze commented 1 year ago

The relevant test should be test_versions, but I tried adding ('pkg~=1.0', ['1.0', '1.1']), to the parametrize list and that didn't make it fail. Maybe you could come up with a variant that does fail without your fix? I also wondered a bit why devpi list works, but that is using the json output and that uses get_versions_filter_iter which isn't affected.

EvaSDK commented 1 year ago

Thanks for the hint, I think I got it covered now.

fschulze commented 1 year ago

Thanks for the follow up. I cherry-picked your commits and changed the test to use BeautifulSoup, the URL class from devpi-common and comparing the whole list of expected versions at once.

devpi / devpi-constrained

Do not stop yielding links matching filter on first match #6