search

python / cpython

The Python programming language
https://www.python.org
Other
63.52k stars 30.43k forks source link

TSan: data race accessing socket fd #116912

Open pitrou opened 8 months ago

pitrou commented 8 months ago

Bug report

Bug description:

The sock_fd member in PySocketSockObject can be mutated while the socket is being used, for example when calling socket.close or socket.detach. However, some other threads can access that member without holding the GIL, and without using atomic instructions. This can trip up Thread Sanitizer in test_asyncio, for example:

0:00:31 load avg: 3.29 [15/52/1] test_asyncio.test_sock_lowlevel worker non-zero exit code (Exit code 66)
==================
WARNING: ThreadSanitizer: data race (pid=7227)
  Write of size 4 at 0x7fd903611570 by thread T19:
    #0 sock_close /home/runner/work/cpython/cpython/./Modules/socketmodule.c:3339:20 (_socket.cpython-313d-x86_64-linux-gnu.so+0xdbde) (BuildId: 7a12da96172d453cd023792c027755908db49f4e)
[...]

  Previous read of size 4 at 0x7fd903611570 by main thread:
    #0 sock_sendto_impl /home/runner/work/cpython/cpython/./Modules/socketmodule.c:4474:29 (_socket.cpython-313d-x86_64-linux-gnu.so+0x131e5) (BuildId: 7a12da96172d453cd023792c027755908db49f4e)
    #1 sock_call_ex /home/runner/work/cpython/cpython/./Modules/socketmodule.c:968:19 (_socket.cpython-313d-x86_64-linux-gnu.so+0x10fa1) (BuildId: 7a12da96172d453cd023792c027755908db49f4e)
    #2 sock_call /home/runner/work/cpython/cpython/./Modules/socketmodule.c:1020:12 (_socket.cpython-313d-x86_64-linux-gnu.so+0xf1a1) (BuildId: 7a12da96172d453cd023792c027755908db49f4e)
    #3 sock_sendto /home/runner/work/cpython/cpython/./Modules/socketmodule.c:4532:9 (_socket.cpython-313d-x86_64-linux-gnu.so+0xf1a1)
[...]

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

pitrou commented 8 months ago

@colesbury @gpshead @mpage @erlend-aasland

corona10 commented 1 month ago

It happens from https://github.com/python/cpython/actions/runs/11160718143/job/31021782828?pr=124914 too Maybe worth to fix it.

corona10 commented 1 month ago 
 ==================
WARNING: ThreadSanitizer: data race (pid=11811)
  Write of size 4 at 0x7f788e470220 by main thread:
    #0 _socket_socket_close_impl /home/runner/work/cpython/cpython/./Modules/socketmodule.c:3369:20 (_socket.cpython-314td-x86_64-linux-gnu.so+0xe6fa) (BuildId: fd621939d2f906d64ec9d9113f22b21082c40173)
    #1 _socket_socket_close /home/runner/work/cpython/cpython/./Modules/clinic/socketmodule.c.h:32:20 (_socket.cpython-314td-x86_64-linux-gnu.so+0xe6fa)
    #2 method_vectorcall_NOARGS /home/runner/work/cpython/cpython/Objects/descrobject.c:447:24 (python+0x27d040) (BuildId: f4b70dfbd84ba1a7db1e594342dc691a3ebbd7b7)
    #3 _PyObject_VectorcallTstate /home/runner/work/cpython/cpython/./Include/internal/pycore_call.h:167:11 (python+0x261c31) (BuildId: f4b70dfbd84ba1a7db1e594342dc691a3ebbd7b7)
    #4 PyObject_Vectorcall /home/runner/work/cpython/cpython/Objects/call.c:327:12 (python+0x263730) (BuildId: f4b70dfbd84ba1a7db1e594342dc691a3ebbd7b7)