Open rgommers opened 2 months ago
For TSAN, I found some meson configuration in the QEMU repo that might be something to start with. It looks like all that's needed is making sure -fsanitize=thread
gets into CFLAGS
and LDFLAGS
.
Oh never mind, meson has built-in support for it via b_sanitize
. For some reason I thought there wasn't support.
https://py-free-threading.github.io looks like a cool resource! Are you planning to announce it at some point?
We did, via a blog post. Is there a spot you would have more easily heard the announcement?
I spent a little time looking at setting up TSAN for NumPy. Unfortunately it's not quite as easy to get working as ASAN. I got it working on my Mac, but because you need to set DYLD_INSERT_LIBRARIES
and system integrity protection means that subprocesses don't inherit that environment variable, many tests fail because of that. Eventually I hit a seg fault in test_dtype.py::TestMonsterType::test_dict_recursion
which I suspect is causing a stack overflow under TSAN. I'll have to look closer.
I tried also on my x86 linux laptop and unfortunately in Linux TSAN has issues with ASLR on newer kernels: https://github.com/google/sanitizers/issues/1716#issuecomment-2010399341
I might be able to get this working with clang 18 on Linux, but that might be annoying to get working on CI since it looks like that version hasn't been packaged yet for Ubuntu.
We did, via a blog post. Is there a spot you would have more easily heard the announcement?
I think an announcement on discuss.python.org would make sense...? 🤔
I might be able to get this working with clang 18 on Linux, but that might be annoying to get working on CI since it looks like that version hasn't been packaged yet for Ubuntu.
It's packaged (also for linux) in conda-forge. ;)
conda install clang_linux-64=18
Or plain clang
if you want the bare binaries without any default flags etc. (which you can of course still override after the environment has been activated)
The suggested plan of attack and documented test strategies on https://py-free-threading.github.io are up-to-date given what we know and the tools we have available. However, the testing strategies for uncovering thread-safety issues are falling short of what we need. Improving this should be a priority.
Ideas that need implementing or protyping:
threading
andconcurrent.futures
in this doc section, and look something like therun_in_parallel
from scikit-image)