search

docker-library / python

Docker Official Image packaging for Python
https://www.python.org/
MIT License
2.54k stars 1.06k forks source link

Is there a 3.13 nogil container? #947

Open impredicative opened 1 month ago

impredicative commented 1 month ago

I want a 3.13 nogil container, also with all optimizations, etc. I assume that the current 3.13 containers use the GIL.

tianon commented 1 month ago

We don't have a "nogil" image currently. For 3.13 we're using the default compilation settings, which AFAIK still results in the GIL. Unfortunately, compiling Python is pretty heavy, so expanding our current support matrix to include new "nogil" variants is also going to be a tough sell. :see_no_evil:

tianon commented 1 month ago

I'm not sure how you got there from what I said -- perhaps I can clarify: we do compile Python (currently ~42 times per architecture, in fact, and we support ~10 architectures where compilation is necessary), and it's very heavy to do so, and adding more variants would require us to do so even more times, so is not something we're willing to consider unless/until it is an officially recommended configuration (ideally the default configuration, but we'd be willing to consider an official statement along the lines of "everyone should try this out").

impredicative commented 1 month ago

nogil is the future of Python. It doesn't make sense to not support it. If this request seems strange, that may be so only because I am the first person to ask for it. Rest assured, there will be many more who will progressively ask for it. The pressure will build. This request is easy to reject now, but only now.

tianon commented 1 month ago

I agree that nogil is the future of Python, and there's nothing strange about the request. If I were completely opposed to the idea in general, I probably would've closed the issue already. It's more a matter of resourcing/prioritization, not desire/understanding. If the demand were actively higher, it would also be a lot easier to justify the added resources/maintenance spend -- the best way for folks to signal that is to react with :+1: on the top post in this thread.

To put that another way, my answer is more accurately "not yet", not "never".

If you'd like to help with the work, figuring out what changes are necessary to our current Dockerfiles would be a really useful first step.

ldeluigi commented 1 month ago

An experimental JIT-enabled build would also be great to have

sekrause commented 3 weeks ago

Instead of prodiving a whole second set of "nogil" Docker images the free-threaded executable could also be part of the normal images just like it is done with the official Windows and macOS installers:

CPython now has experimental support for running in a free-threaded mode, with the global interpreter lock (GIL) disabled. This is an experimental feature and therefore is not enabled by default. The free-threaded mode requires a different executable, usually called python3.13t or python3.13t.exe. Pre-built binaries marked as free-threaded can be installed as part of the official Windows and macOS installers, or CPython can be built from source with the --disable-gil option.

https://docs.python.org/3.13/whatsnew/3.13.html#free-threaded-cpython

So we would have /usr/local/bin/python3.13t in every image and call this to use the free-threaded mode.

Unfortunately that still means that all of Python has to be compiled twice.

impredicative commented 3 weeks ago

@sekrause But that would break a lot of tooling that expects Python to be at the prior file path.

sekrause commented 3 weeks ago

But that would break a lot of tooling that expects Python to be at the prior file path.

The normal paths would still be there, but point to the normal GIL version. So if you don't adjust your tooling everything will be as before.

abebus commented 2 weeks ago

If anyone seeking for some basic image to play with, I've made one: https://github.com/abebus/free-threaded-python-docker-image