resolvelib 0.6.0 and pip vendoring

[uranusjr]

Not sure where best to post this. There are mainly three things I want to do:

• Address the “extras cannot see URL specified in non-extra-ed requirement” issue by passing additional requirements to find_matches() (#56)
• Add a provider hook for tree pruning (#67)
• Provide more useful information to get_preferences() to implement more intelligent requirement ordering (https://github.com/pypa/pip/issues/9187#issuecomment-756737783)

All of these requirement changes to the provider interface, and I would like to do them in one bump from 0.5.x to 0.6. But I also want to introduce each of them to pip separately to keep the PRs clean and real users more easily to test. How would be best to approach this? Should I vendor resolvelib from Git commits, or create some versions (e.g. 0.6.0a1) for each breaking change?

[webknjaz]

All of these requirement changes to the provider interface, and I would like to do them in one bump from 0.5.x to 0.6. But I also want to introduce each of them to pip separately to keep the PRs clean and real users more easily to test. How would be best to approach this?

Have you considered continuously testing against major downstream users? I like how PyCA implemented this for cryptography — they essentially allow projects using it to drop-in test scripts into a designated location in their repo that they then run in their CI: https://github.com/pyca/cryptography/tree/master/.github/downstream.d.

You could have one for pip — the test script would check out the pip repo, vendor the current resolvelib there, and run some of their related tests. Maybe not in PRs but on cron, if there's a need to save resources.

Should I vendor resolvelib from Git commits, or create some versions (e.g. 0.6.0a1) for each breaking change?

This also raises another question: is resolvelib planning to follow some sort of a known versioning scheme (like SemVer — yeah, I know that it requires 1.x.x so before that, does 0.x mean major)? Any guarantees? We're starting to use it in Ansible and so it'd be great to understand how to cap the dist requirement on our side to keep things smooth and avoid running into breaking changes unexpectedly (because we don't intend to vendor resolvelib atm).

[pfmoore]

Should I vendor resolvelib from Git commits, or create some versions (e.g. 0.6.0a1) for each breaking change?

I'm not explicitly against vendoring from specific commits, but versions might be more understandable. Having said that, we might need a discussion on the pip tracker about the vendoring side. I'm not sure we want to release a pip version with a copy of resolvelib vendored from a git commit, and with my RM hat on, I'd like to maintain the principle that we can release from master at any time.

I sort of feel that wanting to have all 3 features in the bump from 0.5 to 0.6 is making more problems than it solves here. What's the justification? Why not just say 0.6 adds the first feature, 0.7 adds the second, and 0.8 adds the third? Versions are cheap, and it allows for a 0.6.1 bugfix if needed...

[pradyunsg]

Imma say this is resolved now -- we're doing it one at a time, since that's what we'll have developer availability for.

If that changes, I think keeping changes separate is a good idea, to avoid making the pip-side changes too complex.

[hswong3i]

Ideally we should reference following changes for resolvelib>=0.6.0:

• https://github.com/sarugaku/resolvelib/commit/69f859b2e3564f278c235ff0c68bb85121df9721#diff-e446b6f71d0d55843250911cd5ad86eba92b084926403635bfe12939152b5199
• https://github.com/notatallshaw/pip/commit/778778c0d3926c8399a650c38c816edc94ea19c2#diff-b0e7a52962f5798cdd1515400d385e2db511662b0127aec85bdf301bdf0cb620