It's difficult to track down template errors

[matthew-carroll]

We're using jinja in our static site generator called Static Shock.

In general we're very happy with this implementation of Jinja. It provided us with a reasonable templating library for our use-cases.

However, there's one perpetual difficulty, which is tracking down the root cause of template generation failures. Most situations where we've misconfigured a template result in error messages and stacktraces that don't help us locate the issue.

Here's an example of an error I just ran into. The reason this error is happening is because I screwed something up, but it's difficult to figure out what I screwed up:

Unhandled exception:
NoSuchMethodError: The getter 'call' was called on null.
Receiver: null
Tried calling: call
#0      Object.noSuchMethod (dart:core-patch/object_patch.dart:38:5)
#1      Context.call (package:jinja/src/context.dart:34:25)
#2      StringSinkRenderer.visitCall (package:jinja/src/renderer.dart:233:20)
#3      Call.accept (package:jinja/src/nodes/expressions.dart:375:20)
#4      StringSinkRenderer.visitInterpolation (package:jinja/src/renderer.dart:700:28)
#5      Interpolation.accept (package:jinja/src/nodes.dart:76:20)
#6      StringSinkRenderer.visitOutput (package:jinja/src/renderer.dart:714:12)
#7      Output.accept (package:jinja/src/nodes.dart:109:20)
#8      Template.renderTo (package:jinja/src/environment.dart:569:10)
#9      Template.render (package:jinja/src/environment.dart:556:5)
#10     JinjaPageRenderer._renderJinjaTemplate (package:static_shock/src/plugins/jinja.dart:237:37)
#11     JinjaPageRenderer.renderContent (package:static_shock/src/plugins/jinja.dart:112:5)
#12     StaticShock._renderPages (package:static_shock/src/static_shock.dart:468:28)
<asynchronous suspension>
#13     StaticShock.generateSite (package:static_shock/src/static_shock.dart:244:5)
<asynchronous suspension>
#14     main (file:///Users/matt/Projects/static_shock/packages/static_shock_docs/bin/static_shock_docs.dart:36:3)
<asynchronous suspension>

Any time I see "null" I know that I probably didn't provide a value for a property that's used in the template. But who knows which template variable that is, or where I screwed up.

It would be a big help if someone could take a pass over this package and ensure that every possible error condition in a template comes with a clear message about what went wrong, and where.

For example, imagine that the above error message said:

Unhandled exception:
MissingTemplateVariableValueError: Tried to inject value for "menuItems" in the template 
called "guides.jinja" but no value was provided for "menuItems".

[ykmnkmi]

Can this solve the problem?

[ykmnkmi]

I will add the template name later.

[matthew-carroll]

Can this solve the problem?

Can you describe how that would help? What do you expect users to do with it?

I think that clear error messages should be provided by default, without doing any extra work.

[ykmnkmi]

By default, undefined variables render as an empty string or throw an error when accessing members. With this setup, users can either throw an error or create an undefined object like in the example above. For example:

import 'package:jinja/jinja.dart';

const String source = '''
{{ user.name }}''';

Object? undefined(String name) {
  throw UndefinedError('$name is not defined.');
}

void main() {
  var environment = Environment(undefined: undefined);
  var template = environment.fromString(source);
  print(template.render());
  // Unhandled exception:
  // UndefinedError: user is not defined.
}

Currently, I don't track which template is active (inlcude, inheritance, blocks, macro calls) and don't save variable locations in the AST to provide more information.

Planned for the next version.

[matthew-carroll]

Here's another example:

Unhandled exception:
TemplateSyntaxError: Unexpected char & at 255
#0      Lexer.scan (package:jinja/src/lexer.dart:462:9)
#1      Lexer.tokenize (package:jinja/src/lexer.dart:478:23)
#2      _SyncStarIterator.moveNext (dart:async-patch/async_patch.dart:560:14)
#3      TokenReader.next (package:jinja/src/reader.dart:58:20)
#4      new TokenReader (package:jinja/src/reader.dart:9:5)
#5      Parser.scan (package:jinja/src/parser.dart:1323:18)
#6      Environment.scan (package:jinja/src/environment.dart:330:37)
#7      Environment.parse (package:jinja/src/environment.dart:337:12)
#8      Environment.fromString (package:jinja/src/environment.dart:352:16)
#9      new Template (package:jinja/src/environment.dart:525:9)
#10     JinjaPageRenderer._renderJinjaTemplate (package:static_shock/src/plugins/jinja.dart:225:22)
#11     JinjaPageRenderer.renderContent (package:static_shock/src/plugins/jinja.dart:111:5)
#12     StaticShock._renderPages (package:static_shock/src/static_shock.dart:520:28)
<asynchronous suspension>
#13     StaticShock.generateSite (package:static_shock/src/static_shock.dart:227:5)
<asynchronous suspension>
#14     main (file:///Users/matt/Projects/flutter_test_robots/doc/website/bin/flutter_test_robots_docs.dart:31:3)
<asynchronous suspension>

There's a variety of these. But I wanted to further make the point that it's pretty much impossible to know what these errors mean, or what a user is supposed to do about it. I end up just randomly changing things until the error goes away. That costs quite a bit of time.

This error, for example, could output at least a fragment of the template code that it failed to parse, so that I have some idea of what I'm looking for.

[matthew-carroll]

I just found myself working through another instance of this friction:

Unhandled exception:
type 'Null' is not a subtype of type 'List<dynamic>' of 'incoming'
#0      _TypeError._throwNew (dart:core-patch/errors_patch.dart:105:68)
#1      Function._apply (dart:core-patch/function_patch.dart:11:71)
#2      Function.apply (dart:core-patch/function_patch.dart:35:12)
#3      Environment.callCommon (package:jinja/src/environment.dart:286:21)
#4      Environment.callFilter (package:jinja/src/environment.dart:298:14)
#5      Context.filter (package:jinja/src/context.dart:74:24)
#6      StringSinkRenderer.visitFilter (package:jinja/src/renderer.dart:314:20)
#7      Filter.accept (package:jinja/src/nodes/expressions.dart:423:20)
#8      StringSinkRenderer.visitFor (package:jinja/src/renderer.dart:520:34)
#9      For.accept (package:jinja/src/nodes/statements.dart:55:20)
#10     StringSinkRenderer.visitOutput (package:jinja/src/renderer.dart:714:12)
#11     Output.accept (package:jinja/src/nodes.dart:109:20)
#12     Template.renderTo (package:jinja/src/environment.dart:569:10)
#13     Template.render (package:jinja/src/environment.dart:556:5)
#14     JinjaPageRenderer._renderJinjaTemplate.<anonymous closure> (package:static_shock/src/plugins/jinja.dart:207:37)
#15     Function._apply (dart:core-patch/function_patch.dart:11:71)
#16     Function.apply (dart:core-patch/function_patch.dart:35:12)
#17     Environment.callCommon (package:jinja/src/environment.dart:286:21)
#18     Context.call (package:jinja/src/context.dart:37:24)
#19     StringSinkRenderer.visitCall (package:jinja/src/renderer.dart:233:20)
#20     Call.accept (package:jinja/src/nodes/expressions.dart:375:20)
#21     StringSinkRenderer.visitInterpolation (package:jinja/src/renderer.dart:700:28)
#22     Interpolation.accept (package:jinja/src/nodes.dart:76:20)
#23     StringSinkRenderer.visitOutput (package:jinja/src/renderer.dart:714:12)
#24     Output.accept (package:jinja/src/nodes.dart:109:20)
#25     Template.renderTo (package:jinja/src/environment.dart:569:10)
#26     Template.render (package:jinja/src/environment.dart:556:5)
#27     JinjaPageRenderer._renderJinjaTemplate (package:static_shock/src/plugins/jinja.dart:234:33)
#28     JinjaPageRenderer.renderContent (package:static_shock/src/plugins/jinja.dart:112:5)

As you can see, these errors are easy to cause, and very difficult to root cause.

I thought that it might help me to print out all the variables that are expected by a given Template to help me track down which value is missing, or is of the wrong type.

But the only thing that Template offers is body, which is appears to be the root of a tree structure. So there doesn't seem to be a convenient way to query all the properties that a Template expects. This further makes it difficult to track down what's missing and where.

[ykmnkmi]

For variables used in the template, do you need something like this?

class Template {
  List<String> get variables;
}

Working on UndefinedError if a variable is not in the template context.

[matthew-carroll]

@ykmnkmi that would be helpful. Though it would probably be useful to get the fully qualified variable path. If the template tries to access {{ package.github.name }} then I'd like to know the full path of "package", "github", "name", so I can track down the exact use. Because the variable "name" might appear in multiple places in the template but within different access paths, e.g., {{ site.name }}, {{ user.name }}, etc.

[ykmnkmi]

I see only the package, site, and user variables. The .github.name and .name are attributes.

[matthew-carroll]

I see only the package, site, and user variables. The .github.name and .name are attributes.

I don't know what that means. But if my template tries to access package.github.name and it doesn't exist, then I need to this package to report the full package.github.name. If you only report name then there will still be issues where I can't tell which lookup failed, and I'll be stuck in this same debugging situation.

[ykmnkmi]

In the {{ package.github.name }} expression, package is a variable that should throw an UndefinedError if it's not defined, and github is an attribute that should throw, for example, a NullAccessingError if package defined but is null.

[matthew-carroll]

I don't know what you mean by "should". What I'm saying is that it's very difficult to track down which template variable is causing the template renderer to blow up. I just want to be able to find the source of a problem and fix it. If you make that possible, then I'm happy. If I still can't find the source of the error, then I'm not happy.

[ykmnkmi]

I mean expected implementation.