TurboGears2 and pytest conflict

[kiilerix]

TurboGears2 and pytest doesn't in all cases play well together. Like when using the from tg import tmpl_context pattern used in the documentation in combination with pytest for running doctests.

pytest's doctest support is (in _mock_aware_unwrap) using py3 inspect.

Inside inspect, _is_wrapper will do an innocent looking: hasattr(f, '__wrapped__')

But if the code under test has un (unused) import of a tg context (such as tg.request), it is no longer so innocent. tg will throw: TypeError: No object (name: context) has been registered for this thread (which in py2 would have caught by hasattr, but not in py3.)

I don't know if it should be solved in pytest (perhaps by not using inspect), in TurboGears (perhaps by having a default context or at least be aware of how the standard library use __wrapped__), or in inspect.

python3 -m venv venv
. venv/bin/activate
pip install TurboGears2==2.4.2 pytest==5.3.5
echo "from tg import tmpl_context" > test.py
py.test --doctest-modules test.py 

============================= test session starts ==============================
platform linux -- Python 3.7.6, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /tmp
collected 0 items / 1 error

==================================== ERRORS ====================================
___________________________ ERROR collecting test.py ___________________________
/usr/lib64/python3.7/doctest.py:932: in find
    self._find(tests, obj, name, module, source_lines, globs, {})
venv/lib64/python3.7/site-packages/_pytest/doctest.py:460: in _find
    self, tests, obj, name, module, source_lines, globs, seen
/usr/lib64/python3.7/doctest.py:991: in _find
    if ((inspect.isroutine(inspect.unwrap(val))
venv/lib64/python3.7/site-packages/_pytest/doctest.py:411: in _mock_aware_unwrap
    return real_unwrap(obj, stop=_is_mocked)
/usr/lib64/python3.7/inspect.py:511: in unwrap
    while _is_wrapper(func):
/usr/lib64/python3.7/inspect.py:505: in _is_wrapper
    return hasattr(f, '__wrapped__') and not stop(f)
venv/lib64/python3.7/site-packages/tg/support/objectproxy.py:19: in __getattr__
    return getattr(self._current_obj(), attr)
venv/lib64/python3.7/site-packages/tg/request_local.py:240: in _current_obj
    return getattr(context, self.name)
venv/lib64/python3.7/site-packages/tg/support/objectproxy.py:19: in __getattr__
    return getattr(self._current_obj(), attr)
venv/lib64/python3.7/site-packages/tg/support/registry.py:72: in _current_obj
    'thread' % self.____name__)
E   TypeError: No object (name: context) has been registered for this thread
=============================== warnings summary ===============================
venv/lib64/python3.7/site-packages/_pytest/doctest.py:418
  /tmp/venv/lib64/python3.7/site-packages/_pytest/doctest.py:418: PytestWarning: Got TypeError('No object (name: context) has been registered for this thread') when unwrapping <tg.request_local.TurboGearsContextMember object at 0x7f83f4911d50>.  This is usually caused by a violation of Python's object protocol; see e.g. https://github.com/pytest-dev/pytest/issues/5080
    PytestWarning,

-- Docs: https://docs.pytest.org/en/latest/warnings.html
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
========================= 1 warning, 1 error in 0.43s ==========================

This might be the core of the 2nd problem mentioned on https://github.com/TurboGears/tg2/issues/117

[amol-]

Given the pretty uncommon chances that a StackedObjectProxy is used as a callable and the even less common chances that it gets decorated, I think that https://github.com/TurboGears/tg2/commit/77132ba119733d4050e7dfac5f631edd0a590e8a is a reasonable work-around to the problem.

I hoped that removing __call__ support from the object would have solved the issue, but it seems that unwrap doesn't check that what is being provided is a callable at all.

[kiilerix]

Yeah, that seems like an OK way to avoid the problem. Thanks.

1. Considering that AttributeError usually has args[0] like module 'os' has no attribute 'foo', perhaps use something more helpful than just foo. Perhaps something as no 'context' has has been registered for this thread and there is thus no attribute 'foo'.

2. But this also leads to the question of whether it just always should fail with AttributeError instead of TypeError.

3. If not always returning AttributeError, then perhaps do it for all __ methods - not just __wrapped__.

4. Can you recommend a workaround for using pytest doctest with existing TG2 versions?

[amol-]

Probably going for all dunder methods makes sense. I'll update the patch.

Regarding how to avoid problem with current version. I think that the issue comes from the fact that it's exploring all variables exposed in modules or something like that. Maybe changing from tg import request to things like import tg and then using tg.request instead of just request might fix the issue. A similar issue has also been reported for Flask (who uses threadlocal objects too) with pytest a few times, even though I think it's the first time I see it in the context of doctest.