Upgrade version requirements and add github actions

[digitalresistor]

Python 3.5/3.6 on x86 hang forever during testing for some reason.

[stevepiercy]

Yeah, I tried to figure it out. No luck. See https://github.com/Pylons/pyramid_jinja2/pull/156#issuecomment-583716321

[digitalresistor]

It's attempting to run:

test_abs_tmpl_extends_missing (pyramid_jinja2.tests.test_it.TestIntegrationWithSearchPath) ... 

[stevepiercy]

What do we need to figure this out? Someone with latest Windows?

[digitalresistor]

Someone with Windows, and some time to dump out where its hanging and figure out how to mitigate it or file bugs with Python.

[utek]

I've identified the place of failure to Jinja2 code.

https://github.com/pallets/jinja/blob/2.11.x/src/jinja2/_compat.py#L28

    def reraise(tp, value, tb=None):
        if value.__traceback__ is not tb:
            raise value.with_traceback(tb)
        raise value

It hangs on raise value.with_traceback(tb). WHen I'm trying to step into it it Python process exits with Segmentation Fault.

Still looking into it. If somebody has idea what to check please let me know.

It happens when the value is raised. Not inside the .with_traceback(tb)

[mmerickel]

Any idea what the context is for the exception? Where it might be raised? The exception itself has __traceback__ as well as it’s name and args which are all helpful if you’re able to poke at it.

[utek]

@mmerickel I'll check Anyway. Going deeper the hanging happens in python's traceback.py file. This while loop is going round and round as tb is never None

def clear_frames(tb):
    "Clear all references to local variables in the frames of a traceback."
    while tb is not None:        
        try:
            tb.tb_frame.clear()
        except RuntimeError:
            # Ignore the exception raised if the frame is still executing.
            pass
        tb = tb.tb_next

[utek]

@mmerickel

ipdb> value
TemplateNotFound('missing/tmpl.jinja2',)
ipdb> tb
<traceback object at 0x06935EB8>
ipdb> tp
<class 'jinja2.exceptions.TemplateNotFound'>

[mmerickel]

import traceback; traceback.print_tb(tb)

[utek]

@mmerickel That's what I was missing. Thank you

So traceback in jinja2 looks ok

ipdb> traceback.print_tb(value.__traceback__)
  File "c:\users\utek\envs\p35\lib\site-packages\jinja2\debug.py", line 163, in fake_traceback
    exec(code, globals, locals)
  File "U:\os\pyramid_jinja2\pyramid_jinja2\tests\templates\extends_missing.jinja2", line 1, in top-level template code
    {% extends "missing/tmpl.jinja2" %}
  File "c:\users\utek\envs\p35\lib\site-packages\jinja2\environment.py", line 1088, in render
    return concat(self.root_render_func(self.new_context(vars)))
  File "U:\os\pyramid_jinja2\pyramid_jinja2\tests\templates\extends_missing.jinja2", line 12, in root
  File "c:\users\utek\envs\p35\lib\site-packages\jinja2\environment.py", line 883, in get_template
    return self._load_template(name, self.make_globals(globals))
  File "c:\users\utek\envs\p35\lib\site-packages\jinja2\environment.py", line 857, in _load_template
    template = self.loader.load(self, name, globals)
  File "c:\users\utek\envs\p35\lib\site-packages\jinja2\loaders.py", line 117, in load
    source, filename, uptodate = self.get_source(environment, name)
  File "U:\os\pyramid_jinja2\pyramid_jinja2\__init__.py", line 250, in get_source
    raise TemplateNotFound(name=template, message=message)

meanwhile inside the traceback.py

> c:\users\utek\appdata\local\programs\python\python35-32\lib\traceback.py(217)clear_frames()
    216             import ipdb; ipdb.set_trace()
--> 217             tb.tb_frame.clear()
    218         except RuntimeError:

ipdb> print_tb(tb, limit=20)
  File "U:\os\pyramid_jinja2\pyramid_jinja2\tests\test_it.py", line 219, in test_abs_tmpl_extends_missing
    render(tpath, {})
  File "c:\users\utek\envs\p35\lib\site-packages\pyramid\renderers.py", line 76, in render
    result = helper.render(value, None, request=request)
  File "c:\users\utek\envs\p35\lib\site-packages\pyramid\renderers.py", line 472, in render
    result = renderer(value, system_values)
  File "U:\os\pyramid_jinja2\pyramid_jinja2\__init__.py", line 265, in __call__
    return template.render(system)
  File "c:\users\utek\envs\p35\lib\site-packages\jinja2\environment.py", line 1090, in render
    self.environment.handle_exception()
  File "c:\users\utek\envs\p35\lib\site-packages\jinja2\environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "c:\users\utek\envs\p35\lib\site-packages\jinja2\_compat.py", line 33, in reraise
    res = value.with_traceback(tb)
  File "U:\os\pyramid_jinja2\pyramid_jinja2\tests\templates\extends_missing.jinja2", line 1, in top-level template code
    {% extends "missing/tmpl.jinja2" %}
  File "U:\os\pyramid_jinja2\pyramid_jinja2\__init__.py", line 265, in __call__
    return template.render(system)
  File "c:\users\utek\envs\p35\lib\site-packages\jinja2\environment.py", line 1090, in render
    self.environment.handle_exception()
  File "c:\users\utek\envs\p35\lib\site-packages\jinja2\environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "c:\users\utek\envs\p35\lib\site-packages\jinja2\_compat.py", line 33, in reraise
    res = value.with_traceback(tb)
  File "U:\os\pyramid_jinja2\pyramid_jinja2\tests\templates\extends_missing.jinja2", line 1, in top-level template code
    {% extends "missing/tmpl.jinja2" %}
  File "U:\os\pyramid_jinja2\pyramid_jinja2\__init__.py", line 265, in __call__
    return template.render(system)
  File "c:\users\utek\envs\p35\lib\site-packages\jinja2\environment.py", line 1090, in render
    self.environment.handle_exception()
  File "c:\users\utek\envs\p35\lib\site-packages\jinja2\environment.py", line 832, in handle_exception
    reraise(*rewrite_traceback_stack(source=source))
  File "c:\users\utek\envs\p35\lib\site-packages\jinja2\_compat.py", line 33, in reraise
    res = value.with_traceback(tb)
  File "U:\os\pyramid_jinja2\pyramid_jinja2\tests\templates\extends_missing.jinja2", line 1, in top-level template code
    {% extends "missing/tmpl.jinja2" %}
  File "U:\os\pyramid_jinja2\pyramid_jinja2\__init__.py", line 265, in __call__
    return template.render(system)
  File "c:\users\utek\envs\p35\lib\site-packages\jinja2\environment.py", line 1090, in render
    self.environment.handle_exception()

[mmerickel]

It seems like that traceback is recursing right? I see the "top-level template code" stack frame from jinja2 several times. So probably need to poke at the test_abs_tmpl_extends_missing test and see why that may be the case. pyramid_jinja2 does some semi-crazy rewriting of stuff to make lookups work relative to the template's location on disk instead of the jinja2 search path and I would suspect that there's something going on in that logic. I'm not sure what to poke at right now though. That being said it might be worth trying with older versions of jinja2 to see if something changed there. Do you think you could try a few versions going back to like 2.9.6 or 2.10? https://pypi.org/project/Jinja2/#history

[utek]

@mmerickel Sure. I'll get to that. Need some time tho. Will be back with more details

[mmerickel]

Thank you @utek, this is so helpful.

[digitalresistor]

\o/ thank you so much @utek for reaching out on Twitter and taking the reins on this! <3

[digitalresistor]

The question I also have is why this is only happening on Python 3.6 and 3.5 and ONLY on x86, but not x64.

[utek]

Ok. More tests. It's passing on Jinja2: 2.10.3; 2.9.6; 2.11.2. Fails on Jinja2: 2.11.0 2.11.1

[utek]

Looks like this was the problem: https://github.com/pallets/jinja/issues/1162

https://github.com/pallets/jinja/pull/1163/files

[utek]

diff --git a/setup.py b/setup.py
index d543130..2d30e5f 100644
--- a/setup.py
+++ b/setup.py
@@ -18,7 +18,7 @@ README = open("README.rst").read()
 CHANGES = open("CHANGES.txt").read()

 requires = [
-    "Jinja2>=2.5.0",
+    "Jinja2>=2.5.0,!=2.11.0,!=2.11.1",
     "MarkupSafe",
     "pyramid>=1.3.0",  # pyramid.path.DottedNameResolver
     "zope.deprecation",

This should fix the issue

[digitalresistor]

Thanks @utek!

I've deleted the project from AppVeyor so it no longer should throw errors up.

[utek]

@bertjwregeer Anytime