Most basic Jinja function is inconsistent / crashes if the template has `{{ obj.property }}` elements, and `obj` is `None` or property does not exist

[schittli]

Hello

Thank you very much for sharing your great work!, it makes life with Python much better.

Intro

As a 'end-user' of Jinja, one would expect this most basic behavior: Regardless of whether one works in the template with {{ variable }} or with {{ obj.property }} elements, Jinja behaves always the same if one of these elements does not exist. If Jinja handles missing references differently, then Jinja becomes tedious because one has to handle both errors in the templates and in the Python code differently.

Describe the bug / summary

1. 🆗 Great: If a template references a {{ variable }} which does not exist, then:
   • Jinja adds variable it to meta.find_undeclared_variables
2. ⛔ Basic behavior broke: If a template references a {{ obj.property }} and property does not exist, then:
   • Jinja raises an TemplateSyntaxError exception
   • and it does not add obj.property it to meta.find_undeclared_variables
3. ⛔ Basic behavior broke: If a template references a {{ obj.property }} and obj does not exist, then
   • Jinja raises an UndefinedError exception
   • and it does not add obj.property it to meta.find_undeclared_variables

Code to replicate the bug

To show the different jinja behavior, I have created this RenderTemplate() function which:

• parses the template
• tests if referenced variables are unknown
• displays unknown variables
• displays the Exception Type

from typing import Dict, Any
import jinja2
from jinja2 import meta

def RenderTemplate(context: Dict[str, Any], tpl: str):
   try:
      # Setup jinja2
      env = jinja2.Environment(undefined = jinja2.DebugUndefined)
      # Create the Template
      template = env.from_string(tpl)
      # Render the Template
      rendered = template.render(context)

      # Test if the Template has unknown variables
      ast = env.parse(rendered)
      undefinedVars = jinja2.meta.find_undeclared_variables(ast)

      # Print the result
      print(f' > {rendered=}')
      if undefinedVars:
         print(f' > {undefinedVars=}')

   except Exception as ex:
      # Print the Exception
      exceptionType = f'{ex.__class__.__module__}.{ex.__class__.__name__}'
      print(f' > {exceptionType=}')
      print(f' > {ex=}')

   return

The 4 tests

Now, we test 4 different situations of non-existent variables and properties. We use those variables / objects:

• message = 'Good morning'
• person = Person(prename = 'John')

#	Template	Error in Template	Rendered Text	Content of meta. find_undeclared_variables()
1	'{{ message }} {{ person.prename }}'	All references are OK	✔️'Good morning John'	✔️Empty
2	'{{ unknwonVar }} {{ person.prename }}'	unknwonVar does not exist	✔️'{{ unknwonVar }} John'	✔️unknwonVar
3	'{{ message }} {{ person.name }}'	Property .namedoes not exists	❌️Exception: TemplateSyntaxError	❌️None
4	'{{ message }} {{ noperson.prename }}'	noperson does not exist	❌️Exception: UndefinedError	❌️None

🙏 Please:

• Change the handling of references that cannot be resolved, so that it always behaves exactly as it already does if a {{ var }} does not exist.
• If this error handling is consistent, then it is much easier for the users of Jinja to handle such typical errors, because they do not have to handle different variants and even exceptions. (In a template engine it is quite normal that variables / objects / properties do not exist, so these errors should not have to be handled as exceptions.)

This code calls all 4 tests:

# A simple Class
class Person:
   def __init__(self, prename):
      self.prename = prename

# Setup Data
message = 'Good morning'
person = Person(prename = 'John')

# Tests
print('\nTest 1: All variables are defined:')
template = '{{ message }} {{ person.prename }}'
print(f' {template=}')
RenderTemplate(locals(), template)
# Result:
#  > rendered='Good morning John'

print('\nTest 2: unknwonVar is used, which does not exist:')
template = '{{ unknwonVar }} {{ person.prename }}'
print(f' {template=}')
RenderTemplate(locals(), template)
# Result:
#  > rendered='{{ unknwonVar }} John'
#  > undefinedVars={'unknwonVar'}

print('\nTest 3: Access person.name, a *Property* which does not exist:')
template = '{{ message }} {{ person.name }}'
print(f' {template=}')
RenderTemplate(locals(), template)
# Result:
#  > exceptionType='jinja2.exceptions.TemplateSyntaxError'
#  > ex=TemplateSyntaxError("expected token 'end of print statement', got 'such'")

print('\nTest 4: We access noperson.name - an *Object* which does not exist:')
template = '{{ message }} {{ noperson.prename }}'
print(f' {template=}')
RenderTemplate(locals(), template)
# Result:
#  > exceptionType='jinja2.exceptions.UndefinedError'
#  > ex=UndefinedError("'noperson' is undefined")

Thanks a lot, kind regards, Thomas

[davidism]

3. ⛔ Basic behavior broke: If a template references a {{ obj.property }} and property does not exist, then:
   • Jinja raises an TemplateSyntaxError exception
   • and it does not add obj.property it to meta.find_undeclared_variables

I can't reproduce raising TemplateSyntaxError. Sounds like you have a syntax error that's not present in the example you've shown.

5. ⛔ Basic behavior broke: If a template references a {{ obj.property }} and obj does not exist, then
   • Jinja raises an UndefinedError exception
   • and it does not add obj.property it to meta.find_undeclared_variables

This and the previous described behavior about find_undeclared_variables is correct. Like Python, Jinja parses the template into an AST, then produces a sequence of instructions to follow at runtime. At parsing, Python and Jinja know nothing about whether any given name will refer to an object that has further accessed attributes. It can know if the name itself is not defined within the template, but not further attribute access.

Here are two Python-only examples that demonstrate this. you can't tell from looking whether they will succeed, you must run them in context. This is because Python has very dynamic ways to work with attributes.

# Here we know what `a` and `b` are, but not whether `.value` would succeed for them.
a = ThingA()
b = ThingB()
a.value + b.value

# Here we don't know what `a` and `b` are, so we don't know if `.value` is valid either.
a.value + b.value

[schittli]

Hello @davidism thank you very much for your answer & help! I will try to re-test the TemplateSyntaxError issue and I will follow up.

Related to:

# Here we don't know what a and b are, so we don't know if .value is valid either. a.value + b.value

I would be surprised if hasattr(object, propertyName) could not check whether property .value exists.

Therefore, there is probably no reason why a preprocessor cannot use the AST parser to check whether all variables are resolvable and thus save all errors in meta.find_undeclared_variables().

Thanks a lot, kind regards, Thomas