Best practices for `requires-python`

TheRealBecks commented 2 months ago

I had some thoughts about version handling and documentation in #7352, but that was out of scope, so I created this issue.

~I used the wrong version identifier in the first place which resulted to 3.12.0~ -> #7426
I did not expect uv to be stuck at 3.12.0 when defining requires-python = "==3.12" and changing it to ~=3.12
I think it's best to change the identifier in the documentation to ~=3.12 (or ==3.12.*?) and describing what that means instead of using >=3.12 which could lead to 3.13.x
Maybe pointing the interested user in a notice field to the Version specifier documentation?

zanieb commented 2 months ago

Using >=3.12 is best practice for requires-python — it's generally not great to add upper version pins to the Python requirement of the project. Instead, you should use uv python pin 3.12 to pin the version.

edmorley commented 2 months ago

@zanieb So this actually overlaps with a topic that affects us quite a bit (both in general, and it's also come up again with the recent addition of Poetry support).

For libraries (or tools/software that is going to be distributed to a variety of end-user machines that needs to support multiple Python versions, such as CLIs, ...) supporting a range of Python versions in requires-python absolutely makes sense.

However, there are a large number of projects (and I would posit a much larger group than the library/... case), for which the project only needs to support, and will likely only ever be capable of supporting (due to lack of CI coverage of all Python versions in a CI matrix), a single major Python version.

For example, when deploying a Django app to a PaaS, the build system needs to pick a single version of Python for the project to install into the container and use to boot the app at launch.

But what version should be picked if there is no .python-version file, and the requires-python field contains say >=3.8? Do we:

Always pick the latest GA Python (3.12)? However, that's not ideal, since the app could break when Python 3.13 is released.
On the first build, pick the current GA Python (3.12), store that in the per-app build cache, and then forever use the same major Python version with that app? However, that's not ideal, since it's hidden state/magic (ie: requires more explanation when the version reaches EOL and you have to tell the user to change a version they didn't even pick/set), and means inconsistency between local development, review apps, staging app and production apps, given state isn't shared.
Pick the lowest Python in the range (so 3.8)? However, that's still not ideal, since it's likely an old Python (particularly since people unlikely to remember to bump requires-python over time).
Fail the build until they add a .python-version file. However, errors aren't ideal for an onboarding experience.

Plus for all of the above apart from (4), there is no guarantee that the Python version picked by the build system will match the end users machine - or even that two developers from the same team are using the same Python version. This then leads to "well it works for me locally, but not on your platform" type support tickets.

The fact that uv init creates a .python-version file (with our preferred, major-version-only syntax) and also validates that the .python-version file's version is compatible with the range in requires-python (when running other commands) is already a big improvement over e.g. Poetry.

However, I imagine there will still be a number of uv-using projects that didn't use uv init and so don't have a .python-version file.

As such, it would be great to either:

Find ways to increase the adoption of .python-version. For example, when locking warn if the file doesn't exist or even automatically create the file (ie: lock the Python version at the same time as the dependencies, which conceptually seems like it might fit?). There would of course need to be a way to opt out, and perhaps it could be limited to when the project was using "app" mode (disabled if package = true etc).
Or, encourage the use of "safer" (and deploy/dev-prod parity friendly) requires-python values for "app" use-cases - such ==3.12.* instead of >=3.12. (Whilst technically ~=3.12.0 would also count as "safe" from a "doesn't pull in major version changes" point of view, IMO the ~= operator is too surprising, given that ~=3.12 and ~=3.12.0 are not equivalent - and so I would prefer to steer end-users away from it.)

Lastly, imagine the scenario where:

someone runs uv init now for their project of type "app", giving a .python-version containing 3.12 and a pyproject.toml with a requires-python of >=3.12
over the years they bump their .python-version version, but don't think about needing to update the requires-python version
eventually Python 3.12 reaches EOL
as the user later adds or updates packages, uv's resolver has to find packages that are still compatible with Python 3.12 (to meet the requires-python range), but this becomes increasingly hard over time
this causes outdated packages, which is confusing for the user since they've requested a modern Python version in their .python-version file and may not even be aware about requires-python and the impact it has on package resolution.

In that scenario, it seems using a wide-version-range value for requires-python (such as the default of >=3.X) is a net-negative for UX for the "app" project type, even if a .python-version file is present. As such, perhaps nudging "app" project types towards a requires-python of e.g. ==3.12.* would still be preferred? (Though an alternative might be for uv to output a warning if the requires-python range includes EOL Python versions perhaps?)

zanieb commented 2 months ago

Thanks for the thorough comment @edmorley — I basically agree there are problems with using >= for unpublished packages. I think it'd be reasonable to use ==X.y.* instead, but I am curious to hear from more of the uv team though.

2-5 commented 2 months ago

I have a related issue, and I'm not sure what's the best approach.

I have a modular monolith, where the same Python package/code can run as multiple roles - webserver or worker for example. Each role has different dependencies, and I'm using project.optional-dependencies for that:

[project]
requires-python = ">=3.11"
dependencies = [
    "prometheus-client>=0.20.0",
]

[project.optional-dependencies]
webserver = [
    "aiohttp>=3.9.5",
]
worker = [
    "orjson>=3.10.7",
]

If I want to run a particular checkout using the worker role I initialize the .venv like this: uv sync --frozen --extra worker

The two pain points I encountered:

there is no way to make uv remember how the .venv was initialized. If after the above command I run uv sync, it will not take into account that it was run previously with --extra worker, and it will change the installed packages
the different roles can have different Python version requirements. webserver is only compatible with 3.11, while worker also works with 3.12. This means I can't use a Git commited .python-version, since it would have to be different between the roles. So instead I do this uv sync --frozen --extra webserver --python 3.11

So I'm wondering if there are some best practices on how to work with separate sets of somewhat incompatible optional dependencies.

Seazs commented 2 months ago

I have a similar issue with the usage of the following type of command:

uv python install '>=3.11'

With the latest pre-release Python version 3.13.0rc2 being available, if no other version is installed, uv will install 3.13.0rc2.

Would it not be better if uv installed the latest stable version (3.12.6) in that scenario?

As explained in PEP440, section Handling of pre-release, "Pre-releases of any kind, including developmental releases, are implicitly excluded from all version specifiers, unless they are already present on the system, explicitly requested by the user, or if the only available version that satisfies the version specifier is a pre-release."

Also, I am new to creating issues or requests, so let me know if I am doing something wrong.

zanieb commented 2 months ago

@Seazs thanks for the report, that's different than this issue — that's a bug. We'll track it in https://github.com/astral-sh/uv/issues/7637

astral-sh / uv

Best practices for `requires-python` #7429