Filters to implement

[Keats]

Intro

Follow the examples in filters/string.rs with tests. Need to add all common and expected filters from Jinja2 and Django. Feedback on the API wanted as well! Each filter should have tests for all the possible options. Tera filters can only be applied to identifers and their arguments have to be named.

Strings

• [x] lower -> lowercase a string
• [x] urlencode -> https://docs.djangoproject.com/en/1.10/ref/templates/builtins/#urlencode
• [x] wordcount -> https://docs.djangoproject.com/en/1.10/ref/templates/builtins/#wordcount
• [ ] truncate -> need to add preserve words option and preserve tags options -> mix of all truncatechars* likehttps://docs.djangoproject.com/en/1.10/ref/templates/builtins/#truncatechars and http://jinja.pocoo.org/docs/dev/templates/#truncate
• [ ] truncatewords: same as truncate but on word level rather than chars
• [x] striptags -> https://docs.djangoproject.com/en/1.10/ref/templates/builtins/#striptags
• [x] slugify -> https://docs.djangoproject.com/en/1.10/ref/templates/builtins/#slugify (https://github.com/django/django/blob/master/django/utils/text.py#L417 to not forget about unicode chars)
• [x] escape -> https://docs.djangoproject.com/en/1.10/ref/templates/builtins/#escape
• [ ] escape_js
• [ ] escape_sql
• [x] capitalize -> first char uppercase, rest lowercase
• [x] replace -> http://jinja.pocoo.org/docs/dev/templates/#replace
• [x] title -> http://jinja.pocoo.org/docs/dev/templates/#title
• [x] addslashes -> https://docs.djangoproject.com/en/1.10/ref/templates/builtins/#addslashes

  Numbers

• [x] filesizeformat -> https://docs.djangoproject.com/en/1.10/ref/templates/builtins/#filesizeformat
• [x] round -> http://jinja.pocoo.org/docs/dev/templates/#round
• [x] pluralize -> https://docs.djangoproject.com/en/1.10/ref/templates/builtins/#pluralize

  Arrays

• [ ] sort -> need to think this one a bit on how to sort
• [x] first -> https://docs.djangoproject.com/en/1.10/ref/templates/builtins/#first
• [x] last -> https://docs.djangoproject.com/en/1.10/ref/templates/builtins/#last
• [x] join -> https://docs.djangoproject.com/en/1.10/ref/templates/builtins/#join

  Common

Those should be in filters/mod.rs or in a filters/common.rs file.

• [x] length -> https://docs.djangoproject.com/en/1.10/ref/templates/builtins/#length applies to string and arrays
• [x] reverse -> http://jinja.pocoo.org/docs/dev/templates/#reverse applies to string and arrays

  Dates

Date filters will wait until Tera has its own serialization instead of using JSON

---

Note: I left out some filters that I don't really the use for like ljust in Django, but happy to include more than the currently listed if they are needed. I also probably forgot some

[Peternator7]

Hey I saw this on this-week-in-rust and thought I might give helping out a shot. I just put in a PR for first,last, and join on the array filters. Let me know if you have any feedback, and if they're okay, I might be able to help with a few others!

[orhanbalci]

@Keats Should title filter respect whitespaces? Documentation does not state anything about whitespaces.

[Keats]

afaik it's something like split on whitespace, uppercase first char of each and join them with " "

Jinja impl: https://github.com/pallets/jinja/blob/8a49e066af3d88d1edb34aa1680b668c959dfcf7/jinja2/filters.py#L183-L190 Jinja tests: https://github.com/pallets/jinja/blob/f4092239019d9a9130dd84ef6278e58061d6bb93/tests/test_filters.py#L201-L223

(Jinja is a bit smarter since it takes into account thinks like <, { that are not letters)

[orhanbalci]

@Keats For escape filter can we use an external crate like https://github.com/veddan/rust-htmlescape

[Keats]

Yeah I've seen of couple on crates.io, need to see which one would be the better fit

[Keats]

I just released 0.2.0 which contains all the filters done so far, thanks @orhanbalci @foophoof @Peternator7 !

[Keats]

Added escape_js and escape_sql to the list, tempted to rename escape to escape_html once autoescape is in Note that I'm not super convinced on escape_sql yet

[clarfonthey]

Was looking at docs and IMO the pluralize filter is kind of confusing and not easily generalisable to multiple languages. I personally think it'd be best to force people to do if statements, because the code is more clear to me to see:

{% if num == 1 %}
    1 message
{% else %}
    {{ num }} messages
{% endif %}

Rather than:

{{ num }} message{{ num | pluralize }}

Especially considering the number of irregular plurals for which this won't work even in English.

Of course, this is just my opinion, but I think that Django having this filter isn't worth adding it for Tera. That's my 2¢. :)

[clarfonthey]

Another comment I have on sort is that it should be split into strsort and numsort where strsort sorts lexically and numsort sorts by a version-based sort (i.e. 12.34.56 > 1.2.3) and then a lexcal sort for any non-numeric characters. I feel like this would be a lot clearer than just a sort.

[Keats]

I agree for the pluralize, it's probably going to get changed when/if I have time for i18n.

For sort, I'm still unsure whether I want to add it at all. The only sort I can see myself doing is a string sort, anything else I'd do it in Rust itself I think

[gyscos]

Adding the "precision" parameter for the round filter would be nice.

[Keats]

@gyscos agreed, opened #99 to track that. PR welcome!

[Keats]

Added precision, it will be in 0.6

[DemiMarie]

It would be awesome to have an HTML sanitizing filter (only allowed in content, not in attributes) that allows only a whitelisted safe subset of HTML (for the given context).

[clarfonthey]

I'm rather against escape_sql because it encourages client-side query sanitisation which seems bad to me.

[Keats]

@clarcharr I added the escape_sql after seeing https://github.com/hashedin/jinjasql but I guess we can just replace the variables by ? or whatever pg/mysql expects for prepared statements.

@DemiMarie how would you know the context? I assume you mean for example li only allowed in ul? If there is a HTML sanitization lib in Rust like https://pypi.python.org/pypi/bleach, it should be easy to write your own but I don't think it would be possible for Tera to know the context

[DemiMarie]

@Keats Such a library does exist (https://crates.io/crates/ammonia). The context would need to be determined by parsing the template.

[Keats]

But then it requires Tera to parse invalid/incomplete HTML (and CSS, JSON etc from #190) which doesn't seem achievable imo

[DemiMarie]

@Keats Servo's parsers are browser-grade and are available as libraries.

[batisteo]

Trying to access or render a variable that doesn't exist will result in an error.

I’d like to see the default filter (and maybe default_if_none).

{{ lang | default(value="en") }}

[Keats]

I believe that should be an easy one to add and would be a "fake" filter, otherwise the expression would fail with lang not being defined if it's not in the context. If anyone wants to add it, please do it in the 0.11 branch or I'll add it myself next week or so

[Keats]

@batisteo I've added a default in the 0.11 branch which is basically the default_if_none from Django. I don't really see the usecase of default over default_if_none so if you have some examples, I'll take them!

[Keats]

Still missing some like truncate but this issue has served its purpose!

[Hugo-Trentesaux]

(still needing the truncate preserve words and preserve tags options)

[Keats]

I don't know about preserve words, it only works for some languages. I don't think jinja2 has an option to preserve tags?

[Hugo-Trentesaux]

I was thinking about the django filter : https://docs.djangoproject.com/en/2.2/ref/templates/builtins/#truncatechars-html By the way, striptags misses the html entities like { and so on. I am planning to have a look on it after I finished migrating my blog from Pelican to Zola.

[stanislavleonchik]

Is anybody know how to use in tera template something like "| floatformat:2" from django?