search

django / asgiref

ASGI specification and utilities
https://asgi.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
1.48k stars 210 forks source link

SyncToAsync no longer thread-sensitive #482

Open markgajdosik opened 1 month ago

markgajdosik commented 1 month ago

3.8.x no longer follows the thread-sensitive principle of a single executor. I have an application that worked without issues on 3.7.2, but since upgrading to Python 3.12, I also upgraded asgiref to 3.8.1 as it adds support for this version of Python. It's a REST API client powered by aiohttp that pulls data into the database.

Despite the fact that every call is thread_sensitive=True, I get deadlocks about 80% of the time. When I log the ID of the executor it wants to use in SyncToAsync.__call__, I am seeing that many executors are being spun up and used to run sync code in an async loop.

I'll try to investigate some more & provide more details, but my time on this ticket is limited and I am leaning towards a perhaps risky downgrade to more stable 3.7.2, even with Python 3.12.

carltongibson commented 1 month ago

@markgajdosik Thanks for the report.

It's not really actionable as is. Some kind of minimal reproduce is probably needed to be able to say anything specific.

Can you try the main branch? Possibly #478 is related.

markgajdosik commented 1 month ago

Hi @carltongibson, I thought you might say that. I now managed to work around the issue by creating a wrapper around SyncToAsync which passes thread_sensitive=False and own single thread executor.

I'll try what you suggested and will think of a way to separate the issue into something you can use to reproduce the problem.