docker-library / python

Docker Official Image packaging for Python
https://www.python.org/
MIT License
2.56k stars 1.07k forks source link

Is there a 3.13 nogil container? #947

Open impredicative opened 2 months ago

impredicative commented 2 months ago

I want a 3.13 nogil container, also with all optimizations, etc. I assume that the current 3.13 containers use the GIL.

tianon commented 2 months ago

We don't have a "nogil" image currently. For 3.13 we're using the default compilation settings, which AFAIK still results in the GIL. Unfortunately, compiling Python is pretty heavy, so expanding our current support matrix to include new "nogil" variants is also going to be a tough sell. :see_no_evil:

tianon commented 2 months ago

I'm not sure how you got there from what I said -- perhaps I can clarify: we do compile Python (currently ~42 times per architecture, in fact, and we support ~10 architectures where compilation is necessary), and it's very heavy to do so, and adding more variants would require us to do so even more times, so is not something we're willing to consider unless/until it is an officially recommended configuration (ideally the default configuration, but we'd be willing to consider an official statement along the lines of "everyone should try this out").

impredicative commented 2 months ago

nogil is the future of Python. It doesn't make sense to not support it. If this request seems strange, that may be so only because I am the first person to ask for it. Rest assured, there will be many more who will progressively ask for it. The pressure will build. This request is easy to reject now, but only now.

tianon commented 2 months ago

I agree that nogil is the future of Python, and there's nothing strange about the request. If I were completely opposed to the idea in general, I probably would've closed the issue already. It's more a matter of resourcing/prioritization, not desire/understanding. If the demand were actively higher, it would also be a lot easier to justify the added resources/maintenance spend -- the best way for folks to signal that is to react with :+1: on the top post in this thread.

To put that another way, my answer is more accurately "not yet", not "never".

If you'd like to help with the work, figuring out what changes are necessary to our current Dockerfiles would be a really useful first step.

ldeluigi commented 2 months ago

An experimental JIT-enabled build would also be great to have

sekrause commented 2 months ago

Instead of prodiving a whole second set of "nogil" Docker images the free-threaded executable could also be part of the normal images just like it is done with the official Windows and macOS installers:

CPython now has experimental support for running in a free-threaded mode, with the global interpreter lock (GIL) disabled. This is an experimental feature and therefore is not enabled by default. The free-threaded mode requires a different executable, usually called python3.13t or python3.13t.exe. Pre-built binaries marked as free-threaded can be installed as part of the official Windows and macOS installers, or CPython can be built from source with the --disable-gil option.

https://docs.python.org/3.13/whatsnew/3.13.html#free-threaded-cpython

So we would have /usr/local/bin/python3.13t in every image and call this to use the free-threaded mode.

Unfortunately that still means that all of Python has to be compiled twice.

impredicative commented 2 months ago

@sekrause But that would break a lot of tooling that expects Python to be at the prior file path.

sekrause commented 2 months ago

But that would break a lot of tooling that expects Python to be at the prior file path.

The normal paths would still be there, but point to the normal GIL version. So if you don't adjust your tooling everything will be as before.

abebus commented 2 months ago

If anyone seeking for some basic image to play with, I've made one: https://github.com/abebus/free-threaded-python-docker-image

martin-budbee commented 2 weeks ago

+1 can confirm; would love to use this both at work and at home!

SDAravind commented 2 weeks ago

Python images without GIL and JIT enabled should be part of docker images. I have been using docker as a dev container and to experiment new versions of python. Thanks to all the work been done so far.