search

Quansight-Labs / free-threaded-compatibility

A central repository to keep track of the status of work on and support for free-threaded CPython (see PEP 703), with a focus on the scientific and ML/AI ecosystem
https://py-free-threading.github.io/
MIT License
158 stars 22 forks source link

Document that hypothesis is thread-unsafe #116

Open ngoldbaum opened 1 week ago

ngoldbaum commented 1 week ago

See https://hypothesis.readthedocs.io/en/latest/details.html#thread-safety-policy and https://github.com/indygreg/python-zstandard/pull/243 for a time this came up in the real world.

ngoldbaum commented 1 week ago

@zac-hd as a hypothesis maintainer I'd appreciate your take on my assessment here.

Zac-HD commented 1 week ago

You're correct that Hypothesis is not threadsafe. I haven't yet worked out what to do about that given free-threading changes; it'd be great to support but also we really do have to manage some global state.

Upstream issue: https://github.com/HypothesisWorks/hypothesis/issues/4028

colesbury commented 4 days ago

Is this mostly a matter of not being able to use tools like pytest-run-parallel with hypothesis or are there other general free-threading issues?

ngoldbaum commented 4 days ago

It's the same as pytest, it's fine to spawn worker threads in a hypothesis test but hypothesis itself has a lot of thread safety issues so if you use constructs provided by hypothesis in a multithreaded context or use hypothesis itself in many threads than you will likely hit issues. Lots of race conditions around global registries and that sort of thing. There's also an on-disk results db that may or may not have thread safety issues.

Zac-HD commented 4 days ago

The database is safe for concurrent use from multiple threads (it's a very basic best-effort KV store), entirely correct on the rest.