Full read-through of the Jinja documentation

[simonw]

I use Jinja enough that I should really do a full read-through of the docs to see what I've missed.

https://jinja.palletsprojects.com/en/3.1.x/

[simonw]

https://jinja.palletsprojects.com/en/3.1.x/intro/ says:

A sandboxed environment can safely render untrusted templates.

I didn't know Jinja had sandboxing!

[simonw]

Templates are compiled to optimized Python code just-in-time and cached, or can be compiled ahead-of-time.

Maybe Datasette should ship compiled-ahead-of-time templates?

[simonw]

https://jinja.palletsprojects.com/en/3.1.x/api/

Jinja uses a central object called the template Environment. Instances of this class are used to store the configuration and global objects, and are used to load templates from the file system or other locations. Even if you are creating templates from strings by using the constructor of Template class, an environment is created automatically for you, albeit a shared one.

[simonw]

from jinja2 import Environment, PackageLoader, select_autoescape
env = Environment(
    loader=PackageLoader("yourapp"),
    autoescape=select_autoescape()
)

This will create a template environment with a loader that looks up templates in the templates folder inside the yourapp Python package (or next to the yourapp.py Python module)

[simonw]

In future versions of Jinja we might enable autoescaping by default for security reasons. As such you are encouraged to explicitly configure autoescaping now instead of relying on the default.

That would be good!

[simonw]

Environment has a bunch of options I'd not seen before - trim_blocks and lstrip_blocks and suchlike.

extensions: List of Jinja extensions to use. This can either be import paths as strings or extension classes

[simonw]

autoescape: As of Jinja 2.4 this can also be a callable that is passed the template name and has to return True or False depending on autoescape should be enabled by default.

I guess that could be a function that returns True for .html files and False for .txt files.

Turns out the select_autoescape() function implements that pattern for you: https://github.com/pallets/jinja/blob/e740cc65d5c54fbebb0f3483add794ec2b47187f/src/jinja2/utils.py#L570-L623

[simonw]

JInja cache size defaults to 400 - that's 400 templates that will have their compiled versions cached. Increased from 50 in Jinja 2.8.

[simonw]

You can customize code_generator_class and context_class though "This should not be changed in most cases".

[simonw]

.overlay(options) is interesting - lets you create a new overlay environment that adds its own customizations: https://jinja.palletsprojects.com/en/3.1.x/api/#jinja2.Environment.overlay

[simonw]

You can return undefined objects with tips to help people debug what went wrong:

if not hasattr(obj, 'attr'):
    return environment.undefined(obj=obj, name='attr', hint='some hint message here')

[simonw]

You can reuse the Jinja expression language like this:

>>> env = Environment()
>>> expr = env.compile_expression('foo == 42')
>>> expr(foo=23)
False
>>> expr(foo=42)
True

Could be interesting to combine this with sandboxing.

[simonw]

.compile_templates() can be used to find all available templates and compile them and put the compiled code optionally in a zip file! https://jinja.palletsprojects.com/en/3.1.x/api/#jinja2.Environment.compile_templates

[simonw]

Both of these work the same:

template.render(knights='that say nih')
template.render({'knights': 'that say nih'})

[simonw]

template.generate() is interesting:

For very large templates it can be useful to not render the whole template at once but evaluate each statement after another and yield piece for piece. This method basically does exactly that and returns a generator that yields one item after another as strings.

Same arguments as render().

https://jinja.palletsprojects.com/en/3.1.x/api/#jinja2.Template.generate

And .stream() "Works exactly like generate() but returns a TemplateStream."

There's also a generate_async(context) equivalent for render_async(context).

[simonw]

>>> t = Template('{% macro foo() %}42{% endmacro %}23')
>>> str(t.module)
'23'
>>> t.module.foo() == u'42'
True

Neat trick for exposing template variables to Python!

[simonw]

TemplateStream:

A template stream works pretty much like an ordinary python generator but it can buffer multiple items to reduce the number of total iterations. Per default the output is unbuffered which means that for every unbuffered instruction in the template one string is yielded.

If buffering is enabled with a buffer size of 5, five items are combined into a new string. This is mainly useful if you are streaming big templates to a client via WSGI which flushes after each iteration.

[simonw]

. is valid in some names!

Filters and tests may contain dots to group filters and tests by topic. For example it’s perfectly valid to add a function into the filter dict and call it to.str

[simonw]

https://jinja.palletsprojects.com/en/3.1.x/api/#the-context

Template filters and global functions marked as pass_context() get the active context passed as first argument and are allowed to access the context read-only.

[simonw]

Filed this:

• https://github.com/simonw/datasette-edit-templates/issues/4

[simonw]

jinja2.MemcachedBytecodeCache - turns out Jinja can cache bytecode in memcached! https://jinja.palletsprojects.com/en/3.1.x/api/#jinja2.MemcachedBytecodeCache

[simonw]

Policies are a bit weird, they're basically a bunch of additional settings for different things: https://jinja.palletsprojects.com/en/3.1.x/api/#policies

e.g. urlize.target and json.dumps_function.

[simonw]

Custom filters: https://jinja.palletsprojects.com/en/3.1.x/api/#custom-filters

def datetime_format(value, format="%H:%M %d-%m-%y"):
    return value.strftime(format)

environment.filters["datetime_format"] = datetime_format

Then:

{{ article.pub_date|datetimeformat }}
{{ article.pub_date|datetimeformat("%B %Y") }}

[simonw]

Tests are a cute feature I didn't know about: https://jinja.palletsprojects.com/en/3.1.x/api/#custom-tests

For example, the test {{ 42 is even }} is called behind the scenes as is_even(42).

[simonw]

Environment.globals are intended for data that is common to all templates loaded by that environment. Template.globals are intended for data that is common to all renders of that template, and default to Environment.globals unless they're given in Environment.get_template(), etc.

OK, so I should mainly use environment globals then.

But:

Environment globals should not be changed after loading any templates

[simonw]

https://jinja.palletsprojects.com/en/3.1.x/api/#low-level-api looks fun:

Environment.lex(*source*, *name=None*, *filename=None*)

Lex the given sourcecode and return a generator that yields tokens as tuples in the form (lineno, token_type, value).

[simonw]

https://jinja.palletsprojects.com/en/3.1.x/api/#the-meta-api has two useful things:

• jinja2.meta.find_undeclared_variables(ast) finds all variables that need to be in the context
• jinja2.meta.find_referenced_templates(ast) lists all imported or included templates

Could have fun with that second one building a debug tool of some sort, maybe even a dot graph visualization.

[simonw]

https://jinja.palletsprojects.com/en/3.1.x/sandbox/ Sandbox actually looks quite good - it's designed to let users craft custom emails for example.

It does have one BIG problem though:

It is possible to construct a relatively small template that renders to a very large amount of output, which could correspond to a high use of CPU or memory. You should run your application with limits on resources such as CPU and memory to mitigate this.

Might still work for Datasette Cloud though, since the only thing users will be hurting if they write a bad template is their own instance.

[simonw]

https://jinja.palletsprojects.com/en/3.1.x/nativetypes/ is interesting - it's designed for non-string-template use cases like config file parsing:

>>> env = NativeEnvironment()
>>> t = env.from_string('{{ x + y }}')
>>> result = t.render(x=4, y=2)
>>> print(result)
6
>>> print(type(result))
int

[simonw]

Doesn't look like you can combine it with sandbox though, which is a shame.

[simonw]

https://jinja.palletsprojects.com/en/3.1.x/templates/ is the bulk of the documentation - "Template Designer Documentation".

[simonw]

Tests have two syntaxes:

{% if loop.index is divisibleby 3 %}
{% if loop.index is divisibleby(3) %}

[simonw]

I should start using this to cleanup my output:

You can also strip whitespace in templates by hand. If you add a minus sign (-) to the start or end of a block (e.g. a For tag), a comment, or a variable expression, the whitespaces before or after that block will be removed:

{% for item in seq -%}
    {{ item }}
{%- endfor %}

This will yield all elements without whitespace between them. If seq was a list of numbers from 1 to 9, the output would be 123456789.

[simonw]

The easiest way to output a literal variable delimiter ({{) is by using a variable expression:

{{ '{{' }}

Or use {% raw %}...{% endraw %}.

[simonw]

https://jinja.palletsprojects.com/en/3.1.x/templates/#template-inheritance

block tags can be inside other blocks such as if, but they will always be executed regardless of if the if block is actually rendered.

And:

If you want to print a block multiple times, you can, however, use the special self variable and call the block with that name:

<title>{% block title %}{% endblock %}</title>
<h1>{{ self.title() }}</h1>
{% block body %}{% endblock %}

[simonw]

I didn't know you could chain super calls:

{{ super.super.super() }}

[simonw]

You can optionally do this for readability:

{% block sidebar %}
    {% block inner_sidebar %}
        ...
    {% endblock inner_sidebar %}
{% endblock sidebar %}

[simonw]

This is interesting - scoped makes the item variable available to the named block from a child template:

{% for item in seq %}
    <li>{% block loop_item scoped %}{{ item }}{% endblock %}</li>
{% endfor %}

[simonw]

You can define a block as MUST be over-ridden:

{% block body required %}{% endblock %}

[simonw]

{% extends layout %}

This lets you pass a compiled template from elsewhere to template.render(layout=xxx).

[simonw]

Jinja functions (macros, super, self.BLOCKNAME) always return template data that is marked as safe.

[simonw]

Two interesting examples: https://jinja.palletsprojects.com/en/3.1.x/templates/#for

<dl>
{% for key, value in my_dict.items() %}
    <dt>{{ key|e }}</dt>
    <dd>{{ value|e }}</dd>
{% endfor %}
</dl>

<dl>
{% for key, value in my_dict | dictsort %}
    <dt>{{ key|e }}</dt>
    <dd>{{ value|e }}</dd>
{% endfor %}
</dl>

[simonw]

I did not know about all of these!

Variable	Description
loop.index	The current iteration of the loop. (1 indexed)
loop.index0	The current iteration of the loop. (0 indexed)
loop.revindex	The number of iterations from the end of the loop (1 indexed)
loop.revindex0	The number of iterations from the end of the loop (0 indexed)
loop.first	True if first iteration.
loop.last	True if last iteration.
loop.length	The number of items in the sequence.
loop.cycle	A helper function to cycle between a list of sequences. See the explanation below.
loop.depth	Indicates how deep in a recursive loop the rendering currently is. Starts at level 1
loop.depth0	Indicates how deep in a recursive loop the rendering currently is. Starts at level 0
loop.previtem	The item from the previous iteration of the loop. Undefined during the first iteration.
loop.nextitem	The item from the following iteration of the loop. Undefined during the last iteration.
loop.changed(val)*	True if previously called with a different value (or not called at all).

[simonw]

Here's how to use cycle:

{% for row in rows %}
    <li class="{{ loop.cycle('odd', 'even') }}">{{ row }}</li>
{% endfor %}

[simonw]

Also this is a way to filter out some items:

{% for user in users if not user.hidden %}
    <li>{{ user.username|e }}</li>
{% endfor %}

No break or continue though - although you can enable them with this extension: https://jinja.palletsprojects.com/en/3.1.x/templates/#loop-controls

[simonw]

<ul>
{% for user in users %}
    <li>{{ user.username|e }}</li>
{% else %}
    <li><em>no users found</em></li>
{% endfor %}
</ul>

Note that, in Python, else blocks are executed whenever the corresponding loop did not break. Since Jinja loops cannot break anyway, a slightly different behavior of the else keyword was chosen.

[simonw]

Wow recursive loops are a thing!

<ul class="sitemap">
{%- for item in sitemap recursive %}
    <li><a href="{{ item.href|e }}">{{ item.title }}</a>
    {%- if item.children -%}
        <ul class="submenu">{{ loop(item.children) }}</ul>
    {%- endif %}</li>
{%- endfor %}
</ul>

Note how loop(item.children) is used to run the same for loop again for the children.

[simonw]

Also this:

{% for entry in entries %}
    {% if loop.changed(entry.category) %}
        <h2>{{ entry.category }}</h2>
    {% endif %}
    <p>{{ entry.message }}</p>
{% endfor %}

[simonw]

{% if %} has elif and else.

[simonw]

I don't think I've ever used Jinja macros: https://jinja.palletsprojects.com/en/3.1.x/templates/#macros

{% macro input(name, value='', type='text', size=20) -%}
    <input type="{{ type }}" name="{{ name }}"
      value="{{ value|e }}" size="{{ size }}">
{%- endmacro %}

<p>{{ input('username') }}</p>
<p>{{ input('password', type='password') }}</p>

Due to how scopes work in Jinja, a macro in a child template does not override a macro in a parent template