search

docker-library / python

Docker Official Image packaging for Python
https://www.python.org/
MIT License
2.5k stars 1.04k forks source link

Threading issues in python3.11 to python3.11.9-slim-bookworm #917

Open taylornelsonbluecargo opened 3 months ago

taylornelsonbluecargo commented 3 months ago

We notice threading issues in our environment with the minor version upgrade

collinc777 commented 3 months ago

Noticing the same thing here. Causing ddtrace-run to fail with gunicorn

jackguac commented 3 months ago

Yep - same problem. Gunicorn workers all crashing. I've downgraded to python3.10 image and all fine.

tianon commented 3 months ago

"threading issues" is not really enough for us to reproduce, let alone help diagnose :wink:

Does anyone have a minimal reproducer?

yosifkit commented 3 months ago

We didn't change any build flags or anything else (except likely some OS package updates from a rebuild) between 3.11.8 and 3.11.9, so if there is an issue, it would be in python itself or a pip installed python package.

LaurentGoderre commented 3 months ago

Does this looks similar to your issues? https://github.com/benoitc/gunicorn/issues/2899

jim-schwoebel commented 2 months ago

having this issue as well, can reproduce on python v 3.11.6

tianon commented 2 months ago

"threading issues" is not really enough for us to reproduce, let alone help diagnose 😉

Does anyone have a minimal reproducer?

Does this looks similar to your issues? benoitc/gunicorn#2899

sanyappc commented 1 month ago

I have a similar problem on virtual machines with the new python3.8-slim-bookworm image, had to switch to python:3.8-slim-bullseye. Error when trying to create a new thread

Traceback (most recent call last):
  File "/venv/lib/python3.8/site-packages/***.py", line 116, in work
    await self.configure_sentry()
  File "/venv/lib/python3.8/site-packages/***.py", line 86, in configure_sentry
    self.sentry_transaction = await get_running_loop().run_in_executor(
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 783, in run_in_executor
    executor.submit(func, *args), loop=self)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 188, in submit
    self._adjust_thread_count()
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 213, in _adjust_thread_count
    t.start()
  File "/venv/lib/python3.8/site-packages/sentry_sdk/integrations/threading.py", line 56, in sentry_start
    return old_start(self, *a, **kw)
  File "/usr/local/lib/python3.8/threading.py", line 852, in start
    _start_new_thread(self._bootstrap, ())
RuntimeError: can't start new thread
tianon commented 1 month ago

"threading issues" is not really enough for us to reproduce, let alone help diagnose 😉 Does anyone have a minimal reproducer?

Does this looks similar to your issues? benoitc/gunicorn#2899

yosifkit commented 1 month ago

If -bullseye works and -bookworm does not, that sounds like libseccomp: https://github.com/docker-library/python/issues/837#issuecomment-1599640563 and more generally, https://github.com/docker-library/official-images/issues/16829

martin-thoma commented 3 days ago

Switching from python:3.11-slim-buster to python:3.11-slim-bookworm I saw the same error message.

edit: it's the libseccomp of the host which you can get via dpkg -l | grep libseccomp. It's NOT the one within the docker container

yosifkit commented 2 days ago

@martin-thoma, it is the libseccomp (and docker and containerd) of the host that you need to update, not anything in the container.

martin-thoma commented 1 day ago

Got it! Updating from docker 20 to docker 24 fixed the issue for me.