tonybaloney / pytest-freethreaded

MIT License
31 stars 3 forks source link

Capture the warnings when you import a module which does not declare no-GIL support #8

Open tonybaloney opened 1 month ago

tonybaloney commented 1 month ago

When you import a C extension that does not support no-GIL, it will raise a warning in CPython, see https://py-free-threading.github.io/porting/

We should capture these warnings and log them in the tests.

Example:

python -m pip install markupsafe
python
>>> import markupsafe
<frozen importlib._bootstrap>:488: RuntimeWarning: The global interpreter lock (GIL) has been enabled to load module 'markupsafe._speedups', which has not declared that it can run safely without the GIL. To override this behavior and keep the GIL disabled (at your own risk), run with PYTHON_GIL=0 or -Xgil=0.

If you made this as a test:


def test_import_warning():
   import markupsafe

Should fail the test because it raised the RuntimeWarning, see https://docs.pytest.org/en/7.1.x/how-to/capture-warnings.html#warns

NCLI commented 1 month ago

Something like this?

@pytest.hookimpl
def pytest_runtest_call(item: pytest.Item):
    # Try item.runtest()
    runtime_err_val = "The global interpreter lock (GIL) has been enabled to load module"
    logger.debug("Running test %s", item.name)

    executor = ThreadPoolExecutor(max_workers=20)
    results = list(executor.submit(get_one_result, [item] * 200))
    exceptions = [isinstance(r, Exception) for r in results]
    filtered_exceptions = []
    for e in exceptions:
        if hasattr(e, message) and runtime_err_val in e.message:
            logger.error(e)
            return results[0]
        else:
            filtered_exceptions.append(e)

    if all(exceptions):
        raise results[0]
    if all(not e for e in exceptions):
        return results[0]
    raise Exception("Result discrepancy") from next(r for r in results if isinstance(r, Exception))
tonybaloney commented 1 month ago

see https://github.com/tonybaloney/pytest-freethreaded/blob/example_code/src/pytest_freethreaded/plugin.py#L52-L64

tonybaloney commented 1 month ago

https://github.com/tonybaloney/pytest-azurepipelines/blob/master/pytest_azurepipelines.py#L208-L209

NCLI commented 1 month ago

This catches and logs the correct warning now, but haven't managed to figure out how to convert them to errors: https://github.com/tonybaloney/pytest-freethreaded/blob/nogil_modules