HypothesisWorks / hypothesis

Hypothesis is a powerful, flexible, and easy to use library for property-based testing.
https://hypothesis.works
Other
7.54k stars 585 forks source link

ResourceWarning unclosed socket when running together with pytest-xdist on windows #4130

Open jenshnielsen opened 5 days ago

jenshnielsen commented 5 days ago

Run the following no op test on windows using pytest + pytest-xdist with workers enabled triggers a Resource warning

import hypothesis.strategies as hst
from hypothesis import given

@given(n_points_pws=hst.integers(min_value=1, max_value=500))
def test_pytest_min_warning(
    n_points_pws,
) -> None:
    pass

when run as

pytest -n 2 test_file.py
Test session starts (platform: win32, Python 3.12.7, pytest 8.3.3, pytest-sugar 1.0.0)
rootdir: D:\source\repos\Qcodes
configfile: pyproject.toml
plugins: sugar-1.0.0, hypothesis-6.112.4, asyncio-0.24.0, cov-5.0.0, mock-3.14.0, rerunfailures-14.0, xdist-3.6.1
asyncio: mode=Mode.STRICT, default_loop_scope=function
2 workers [1 item]

 other_tests/test_simply.py ✓                                                                                         100% ██████████Exception ignored in: <socket.socket fd=720, family=2, type=1, proto=0, laddr=('127.0.0.1', 56571), raddr=('127.0.0.1', 56572)>
Traceback (most recent call last):
  File "C:\Users\jenielse\AppData\Roaming\uv\data\python\cpython-3.12.7-windows-x86_64-none\Lib\threading.py", line 1016, in run
    del self._target, self._args, self._kwargs
                      ^^^^^^^^^^
ResourceWarning: unclosed <socket.socket fd=720, family=2, type=1, proto=0, laddr=('127.0.0.1', 56571), raddr=('127.0.0.1', 56572)>
Zac-HD commented 5 days ago

Hmm, I'm not fully convinced that this is a Hypothesis bug (vs e.g. pytest-xdist); I'd also like to see this occur in a virtualenv with no other packages installed to ensure that it's really just the interaction of Hypothesis and pytest-xdist. Have you tried reproducing on other Python versions? From checking Qcode's CI logs it's not happening on Ubuntu, so I guess it is a Windows-specific issue.

Regardless though, thanks for reporting the issue, wherever it ends up we can't fix issues we don't know about 😁

jenshnielsen commented 4 days ago

I did a bit more digging. It seems that this only happens provided that both pytest-xdist and pytest-rerunfailures is installed. I can reproduce it in an environment that contains the following packages:

Package              Version
-------------------- -------
attrs                24.2.0
colorama             0.4.6
execnet              2.1.1
hypothesis           6.113.0
iniconfig            2.0.0
packaging            24.1
pluggy               1.5.0
pytest               8.3.3
pytest-rerunfailures 14.0
pytest-xdist         3.6.1
sortedcontainers     2.4.0

Provided that I configure pytest to consider errors as warnings

[tool.pytest.ini_options]

filterwarnings = ['error']

and enable at least two xdist workers

pytest .\test_socket_warning.py -n 2

This only happens on windows and yes I agree that its not clear which of the three packages are triggering this.