1.3.4 breaks includes with two variables

[vain]

Consider this example:

#!/usr/bin/env python3

from mako.template import Template
from mako.lookup import TemplateLookup

mylookup = TemplateLookup(directories=['.'])
mytemplate = Template('<%include file="${foo}${bar}"/>', lookup=mylookup)
print(mytemplate.render(foo='foo', bar='bar'))

The content of the foobar file does not matter:

$ cat foobar 
This is the included template.

With 1.3.4, I get the following exception:

(env)$ ./test.py 
Traceback (most recent call last):
  File "/tmp/tmp/env/lib/python3.12/site-packages/mako/pyparser.py", line 36, in parse
    return _ast_util.parse(code, "<unknown>", mode)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmp/env/lib/python3.12/site-packages/mako/_ast_util.py", line 91, in parse
    return compile(expr, filename, mode, PyCF_ONLY_AST)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<unknown>", line 1
    foo}${bar
       ^
SyntaxError: unmatched '}'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/tmp/tmp/./test.py", line 9, in <module>
    mytemplate = Template('<%include file="${foo}${bar}"/>', lookup=mylookup)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmp/env/lib/python3.12/site-packages/mako/template.py", line 299, in __init__
    (code, module) = _compile_text(self, text, filename)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmp/env/lib/python3.12/site-packages/mako/template.py", line 676, in _compile_text
    source, lexer = _compile(
                    ^^^^^^^^^
  File "/tmp/tmp/env/lib/python3.12/site-packages/mako/template.py", line 656, in _compile
    node = lexer.parse()
           ^^^^^^^^^^^^^
  File "/tmp/tmp/env/lib/python3.12/site-packages/mako/lexer.py", line 244, in parse
    if self.match_tag_start():
       ^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmp/env/lib/python3.12/site-packages/mako/lexer.py", line 312, in match_tag_start
    self.append_node(parsetree.Tag, keyword, attributes)
  File "/tmp/tmp/env/lib/python3.12/site-packages/mako/lexer.py", line 129, in append_node
    node = nodecls(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmp/env/lib/python3.12/site-packages/mako/parsetree.py", line 250, in __call__
    return type.__call__(cls, keyword, attributes, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmp/env/lib/python3.12/site-packages/mako/parsetree.py", line 380, in __init__
    super().__init__(
  File "/tmp/tmp/env/lib/python3.12/site-packages/mako/parsetree.py", line 299, in __init__
    self._parse_attributes(expressions, nonexpressions)
  File "/tmp/tmp/env/lib/python3.12/site-packages/mako/parsetree.py", line 330, in _parse_attributes
    code = ast.PythonCode(
           ^^^^^^^^^^^^^^^
  File "/tmp/tmp/env/lib/python3.12/site-packages/mako/ast.py", line 42, in __init__
    expr = pyparser.parse(code.lstrip(), "exec", **exception_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmp/env/lib/python3.12/site-packages/mako/pyparser.py", line 38, in parse
    raise exceptions.SyntaxException(
mako.exceptions.SyntaxException: (SyntaxError) unmatched '}' (<unknown>, line 1) ('foo}${bar') at line: 1 char: 1

Changing it to this works fine, though:

mytemplate = Template('<%include file="${foo}"/>', lookup=mylookup)
print(mytemplate.render(foo='foobar'))

Maybe related to https://github.com/sqlalchemy/mako/pull/397?

Thanks!

[zzzeek]

Yes I was kind of concerned about this new change, but we couldnt get anything to show it failing. I'm going to revert it right now and we can add this new case as a test.

1.3.4 is yanked

[sqla-tester]

Mike Bayer has proposed a fix for this issue in the main branch:

Revert "Update parsetree.py removed "?" from for x in re.compile(r"(\${.+})" …" https://gerrit.sqlalchemy.org/c/sqlalchemy/mako/+/5304

[zzzeek]

thanks for the quick reporting, 1.3.5 is released

[vain]

Alright, thanks for the fix! :)