Python 3: TypeError (no context) which was not present on Python 2

[patrickdepinguin]

The Kallithea project is migrating from Python 2 to Python 3. The codebase itself is ready and works fine on Python 3. But we have two test related issues.

1. First issue happens when the test suite initializes and tries to create a user in the database. At this point the application is not yet loaded, but one of the methods 'get_current_authuser' tries to read something from tmpl_context. In the Python 2 codebase, this worked fine, while in Python 3 it doesn't. Stacktrace is:

../venv/kallithea-review/lib/python3.6/site-packages/py/_path/common.py:383: in visit
    for x in Visitor(fil, rec, ignore, bf, sort).gen(self):
../venv/kallithea-review/lib/python3.6/site-packages/py/_path/common.py:435: in gen
    for p in self.gen(subdir):
../venv/kallithea-review/lib/python3.6/site-packages/py/_path/common.py:424: in gen
    dirs = self.optsort([p for p in entries
../venv/kallithea-review/lib/python3.6/site-packages/py/_path/common.py:425: in <listcomp>
    if p.check(dir=1) and (rec is None or rec(p))])
../venv/kallithea-review/lib/python3.6/site-packages/_pytest/main.py:667: in _recurse
    ihook = self.gethookproxy(dirpath)
../venv/kallithea-review/lib/python3.6/site-packages/_pytest/main.py:482: in gethookproxy
    my_conftestmodules = pm._getconftestmodules(fspath)
../venv/kallithea-review/lib/python3.6/site-packages/_pytest/config/__init__.py:424: in _getconftestmodules
    mod = self._importconftest(conftestpath.realpath())
../venv/kallithea-review/lib/python3.6/site-packages/_pytest/config/__init__.py:474: in _importconftest
    self.consider_conftest(mod)
../venv/kallithea-review/lib/python3.6/site-packages/_pytest/config/__init__.py:527: in consider_conftest
    self.register(conftestmodule, name=conftestmodule.__file__)
../venv/kallithea-review/lib/python3.6/site-packages/_pytest/config/__init__.py:321: in register
    ret = super(PytestPluginManager, self).register(plugin, name)
../venv/kallithea-review/lib/python3.6/site-packages/pluggy/manager.py:127: in register
    hook._maybe_apply_history(hookimpl)
../venv/kallithea-review/lib/python3.6/site-packages/pluggy/hooks.py:333: in _maybe_apply_history
    res = self._hookexec(self, [method], kwargs)
../venv/kallithea-review/lib/python3.6/site-packages/pluggy/manager.py:93: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
../venv/kallithea-review/lib/python3.6/site-packages/pluggy/manager.py:87: in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
kallithea/tests/conftest.py:74: in pytest_configure
    create_test_env(TESTS_TMP_PATH, context.config())
kallithea/tests/fixture.py:372: in create_test_env
    dbmanage.create_default_user()
kallithea/lib/db_manage.py:399: in create_default_user
    lastname=u'User')
kallithea/model/user.py:116: in create_or_update
    cur_user = getattr(get_current_authuser(), 'username', None)
kallithea/lib/utils2.py:484: in get_current_authuser
    if hasattr(tmpl_context, 'authuser'):
../venv/kallithea-review/lib/python3.6/site-packages/tg/support/objectproxy.py:19: in __getattr__
    return getattr(self._current_obj(), attr)
../venv/kallithea-review/lib/python3.6/site-packages/tg/request_local.py:240: in _current_obj
    return getattr(context, self.name)
../venv/kallithea-review/lib/python3.6/site-packages/tg/support/objectproxy.py:19: in __getattr__
    return getattr(self._current_obj(), attr)
../venv/kallithea-review/lib/python3.6/site-packages/tg/support/registry.py:72: in _current_obj
    'thread' % self.____name__)
E   TypeError: No object (name: context) has been registered for this thread

We can workaround it via the changes in kallithea/lib/utils2.py in this commit: https://kallithea-scm.org/repos/kallithea-incoming/changeset/490ef571d8dce7fb74f9c7c0bad980f0a47adaf8

2. After this workaround, we get issues with doctests. By default we run the doctests via pytest too. But in fact, the same error is present when using python -m doctest directly, thus ruling out the pytest dependency.

> python -m doctest kallithea/config/routing.py
Traceback (most recent call last):
  File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib64/python3.6/doctest.py", line 2787, in <module>
    sys.exit(_test())
  File "/usr/lib64/python3.6/doctest.py", line 2777, in _test
    failures, _ = testmod(m, verbose=verbose, optionflags=options)
  File "/usr/lib64/python3.6/doctest.py", line 1950, in testmod
    for test in finder.find(m, name, globs=globs, extraglobs=extraglobs):
  File "/usr/lib64/python3.6/doctest.py", line 933, in find
    self._find(tests, obj, name, module, source_lines, globs, {})
  File "/usr/lib64/python3.6/doctest.py", line 992, in _find
    if ((inspect.isroutine(inspect.unwrap(val))
  File "/usr/lib64/python3.6/inspect.py", line 512, in unwrap
    while _is_wrapper(func):
  File "/usr/lib64/python3.6/inspect.py", line 503, in _is_wrapper
    return hasattr(f, '__wrapped__')
  File "/home/tdescham/repo/contrib/kallithea/venv/kallithea-review/lib/python3.6/site-packages/tg/support/objectproxy.py", line 19, in __getattr__
    return getattr(self._current_obj(), attr)
  File "/home/tdescham/repo/contrib/kallithea/venv/kallithea-review/lib/python3.6/site-packages/tg/request_local.py", line 240, in _current_obj
    return getattr(context, self.name)
  File "/home/tdescham/repo/contrib/kallithea/venv/kallithea-review/lib/python3.6/site-packages/tg/support/objectproxy.py", line 19, in __getattr__
    return getattr(self._current_obj(), attr)
  File "/home/tdescham/repo/contrib/kallithea/venv/kallithea-review/lib/python3.6/site-packages/tg/support/registry.py", line 72, in _current_obj
    'thread' % self.____name__)
TypeError: No object (name: context) has been registered for this thread

Here, it is Python itself that tries to read an attribute, and this gets caught by TG2 objectproxy. Also this worked fine in Python2.

Could you please help with this problem?

To reproduce, first pull the latest development code. Assuming you already have a clone of Kallithea (https://kallithea-scm.org/repos/kallithea):

hg pull https://kallithea-scm.org/repos/kallithea-incoming -r 3ddc91818b3c
hg update 3ddc91818b3c

then proceed with the setup:

virtualenv -p python3 ../kallithea-venv
source ../kallithea-venv/bin/activate
pip install --upgrade pip setuptools
pip install --upgrade -e . -r dev_requirements.txt

Note that at revision 3ddc91818b3c the workaround in kallithea/lib/utils2.py is already in place. If you want to reproduce issue 1, it should be reverted first:

hg backout --no-commit 490ef571d8dc
pytest

For issue 2, you don't necessarily need to revert that workaround, and can just run:

python -m doctest  kallithea/config/routing.py

Thanks in advance.

[amol-]

For case 1 I'm surprised it worked at all on PY2. The normal behaviour I'd expect trying to access tmpl_context outside of a request is that it would complain that there is no context. The only object accessible outside of requests is tg.app_globals, all the other global objects are per-request and usually require the practices described in https://turbogears.readthedocs.io/en/latest/turbogears/testing.html#testing-outside-controllers to work.

Case 2 instead is more interesting, I'm not sure why it's trying to access a request object as a decorator.

I'll have to checkout kallithea codebase locally and check what's going on.

[amol-]

For inspect.unwrap this is being fixed in #118 For the case where Kallithea is explicitly calling hasattr instead I think that the proper fix is to trap the TypeError in Kallithea itself.

So closing this one