Feature Request: Have `packse` support every version of Python that it's scenarios run against

[notatallshaw]

Same motivation as https://github.com/astral-sh/packse/issues/163 but with a different solution.

For a simpler test environment for pip I am trying to avoid setting up multiple Python environment versions, packse requires Python 3.12+ so I am only running tests with a Python 3.12 environment.

Most scenarios in packse are set to Python 3.8, for most scenarios this is not relevant to the results of the test. However for a few scenarios it is actually important to resolve the scenario against the Python version specified.

One solution is I could set the pip tests to only run on the Python version that the scenario is set against. It would make testing locally awkward but it would simplify the testing solution itself.

[notatallshaw]

Thinking about it this would be preferred solution to my problem over #163, and it doesn't mean that packse has to support versions of Python older than 3.12, the scenarios themselves could be updated to be a minimum of Python 3.12.

[zanieb]

Some of the scenarios depend on multiple specific Python versions being available on your path. This is something we're looking to improve handling of over in uv too because there are complaints from package maintainers about having to install specific versions. I'm interested in mocking the Python interpreter version somehow since all the scenarios are agnostic to the actual interpreter used for builds.

[notatallshaw]

Hmm, can you give an example please. Once I identified this is a general issue with these tests I didn't look at the nuances of individual tests.

Once I have a chance (probably not till later in the week) I will consider what is required to make those kinds of tests work with pip. Pip does distribute as a zipapp and it may be valid to consider those types of scenarios for pip in that use case.

[zanieb]

e.g. in https://github.com/astral-sh/packse/blob/bdeca1a73e1c04e998fba67a2bec6291cacd83da/scenarios/requires-python.json#L434-L463 we are testing behavior with specific patch versions of Python

[notatallshaw]

Ah I see, this would be when the environment and the resolver options would be a different version of Python.

Yes I would need to come up with a solution for that other than what I've outlined in this issue.

I don't think there's going to be one solution though that completely fits pip's test suite onto packse. Some tests may just have to be excluded, at least initially.

Like I said, I'll consider it more carefully later this week.

[notatallshaw]

It still seems to be that example doesn't require two versions on the path, it just requires the resolver to be defaulted to one Python (which generally for pip means the Python it is bundeled or installed with) and given the resolver option to resolve like it was resolving a different version of Python.

Given that, I beleive, the easiest way for pip to support the most amount of existing test cases from packse's current scenarios is to impelement this feature request, which will cover all scenarios where a specific version of Python is required, and to implement https://github.com/astral-sh/packse/issues/163 where pip could then be on a specific version of Python and be passed --python-version like it was resolving a different version of Python.

But let me know if I am missing anything.

[zanieb]

I guess I don't see any reason to bump to Python 3.12 for the vast majority of the scenarios.

I'm unsure if this is a great solution to your problem though as we might need some Python version specific scenarios in the future. There's no real guarantee that a single Python version will satisfy our test coverage.

We could easily make packse support Python 3.8+, I just wrote it for 3.12 since it's intended to be more of an application than a library.