ResourceWarning from pytest with responses 0.24.0

[jwodder]

Describe the bug

Consider the following pytest test file:

import requests
import responses

@responses.activate
def test_function():
    responses.get("http://www.example.com", body="Hello!")
    with requests.Session() as s:
        r = s.get("http://www.example.com")
        r.raise_for_status()
        assert r.text == "Hello!"

This is accompanied by the following pytest configuration in tox.ini:

[pytest]
filterwarnings = error  # Highly recommended in general

Running pytest on the above test succeeds without a problem with responses 0.23.3, but if responses 0.24.0 is used, the test fails with:

============================= test session starts ==============================
platform darwin -- Python 3.11.6, pytest-7.4.3, pluggy-1.3.0
rootdir: /Users/jwodder/work/dev/tmp/responses-bug
configfile: tox.ini
collected 1 item

test_case.py F                                                           [100%]

=================================== FAILURES ===================================
________________________________ test_function _________________________________

cls = <class '_pytest.runner.CallInfo'>
func = <function call_runtest_hook.<locals>.<lambda> at 0x1100e80e0>
when = 'call'
reraise = (<class '_pytest.outcomes.Exit'>, <class 'KeyboardInterrupt'>)

    @classmethod
    def from_call(
        cls,
        func: "Callable[[], TResult]",
        when: "Literal['collect', 'setup', 'call', 'teardown']",
        reraise: Optional[
            Union[Type[BaseException], Tuple[Type[BaseException], ...]]
        ] = None,
    ) -> "CallInfo[TResult]":
        """Call func, wrapping the result in a CallInfo.

        :param func:
            The function to call. Called without arguments.
        :param when:
            The phase in which the function is called.
        :param reraise:
            Exception or exceptions that shall propagate if raised by the
            function, instead of being wrapped in the CallInfo.
        """
        excinfo = None
        start = timing.time()
        precise_start = timing.perf_counter()
        try:
>           result: Optional[TResult] = func()

.nox/new_responses/lib/python3.11/site-packages/_pytest/runner.py:341: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.nox/new_responses/lib/python3.11/site-packages/_pytest/runner.py:262: in <lambda>
    lambda: ihook(item=item, **kwds), when=when, reraise=reraise
.nox/new_responses/lib/python3.11/site-packages/pluggy/_hooks.py:493: in __call__
    return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
.nox/new_responses/lib/python3.11/site-packages/pluggy/_manager.py:115: in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
.nox/new_responses/lib/python3.11/site-packages/_pytest/unraisableexception.py:88: in pytest_runtest_call
    yield from unraisable_exception_runtest_hook()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    def unraisable_exception_runtest_hook() -> Generator[None, None, None]:
        with catch_unraisable_exception() as cm:
            yield
            if cm.unraisable:
                if cm.unraisable.err_msg is not None:
                    err_msg = cm.unraisable.err_msg
                else:
                    err_msg = "Exception ignored in"
                msg = f"{err_msg}: {cm.unraisable.object!r}\n\n"
                msg += "".join(
                    traceback.format_exception(
                        cm.unraisable.exc_type,
                        cm.unraisable.exc_value,
                        cm.unraisable.exc_traceback,
                    )
                )
>               warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))
E               pytest.PytestUnraisableExceptionWarning: Exception ignored in: <socket.socket fd=-1, family=2, type=1, proto=0>
E               
E               Traceback (most recent call last):
E                 File "/usr/local/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/socket.py", line 777, in close
E                   self._sock = None
E                   ^^^^^^^^^^
E               ResourceWarning: unclosed <socket.socket fd=14, family=2, type=1, proto=0, laddr=('0.0.0.0', 0)>

.nox/new_responses/lib/python3.11/site-packages/_pytest/unraisableexception.py:78: PytestUnraisableExceptionWarning
=========================== short test summary info ============================
FAILED test_case.py::test_function - pytest.PytestUnraisableExceptionWarning:...
============================== 1 failed in 0.35s ===============================

I suspect that the newer responses isn't closing a socket somewhere, possibly due to failure to use a context manager.

Additional context

No response

Version of responses

0.24.0

Steps to Reproduce

I just filled that out above.

Expected Result

No errors.

Actual Result

See errors above.

[beliaev-maksim]

I think this might be related to #685

I will investigate later

[arthurzam]

This issue most likely also is the cause of the test suite failing on pypy3 (we tested it for packaging responses-0.24.0 for Gentoo). It fails with resource exhaustion of open files.

[viotti]

We don't need – actually don't want – any kind of networking during our test battery execution. The following fixture is a workaround.

import pytest
import unittest

@pytest.fixture(autouse=True)
def disable_builtin_socket(scope='session'):
    with unittest.mock.patch('socket.socket'):
        yield