formating of dates, numbers and more

[phloe]

I've been discussing the viability of mustache as a templating solution with a colleague and he raised some questions about a lacking key feature; formating. And I tend to agree with him - having this as part of the mustache spec would be very helpful.

The string representation of a date or time, the numbers of decimals of a price or the number of characters shown shouldn't be part of your viewmodel - it should (in a perfect world) be handled by the templating language.

I'm thinking something along the lines of being able to define a "format" in some form immediately after the variable name:

{
    price: 2.7532
}

{{price:0.00}}USD

2.75USD

I know JSON doesn't support a native date representation - maybe just treat an ISO datestring as a date object and present it through a format like php's date function (other formats might be more fitting/standard)?

I'm sorry if this issue has been raised a thousand times (it seems so obvious somehow) - but I haven't been able to find an answer to this yet ;D

[janl]

Would this work for you?

{ price: 27532, price_format: function() { // implement format_price(); return format_price(this.price); } }

and: {{format_price}}

PS: don't use floats for decimals, especially money.

[phloe]

That would mean I would "pollute" the viewmodel with a lambda - of which I'm not too keen... :)

And if I'm using the same templates in multiple languages across server/clientside I would also have to have implementations of these lambdas in those too (and maintain them). Then the lustre of mustache quickly wears off :(

Mustache could be the silver bullet :)

Floats/decimals was just a bad example :D

[janl]

I like the idea of helpers, hence my mentioning them in http://writing.jan.io/mustache-2.0.html and my "helpers" branch here: https://github.com/janl/mustache.js/tree/helpers a helper would implement your number format, but you'd still implement them server-side and client-side.

fwiw, polluting view objects with lambdas is rather cool I came to appreciate even though it feels dirty.

[phloe]

Ah, I see.

I like the separation that helpers provides - definately more "separation of concerns" than lambdas ;D

Is it going in the spec anytime soon?

[groue]

This request is so common I had to provide with a solution for my Objective-C implementation. Three goals : no alteration of the Mustache syntax itself, no cooperation ("pollution") of the viewmodel, and provide an expressive-enough API so that users can hook in missing features we haven't thought of yet. https://github.com/groue/GRMustache/blob/master/Guides/sample_code/number_formatting.md

TL;DR : by providing carefully crafted hooks in the Mustache rendering engine, we can let users extend Mustache in many ways, and keep the "kernel" clean and simple.

[janl]

basically what @groue says, all we need to get is an update to the spec and convince the people involved with it :)

[davidsantiago]

Just for discussion, two other ways I've seen of doing this are in ctemplate's template modifiers and Liquid's filters. They're both pretty similar to each other.

[groue]

A clarification : as I said, my goal was to provide a solution which does not alter the Mustache syntax. My goal was to provide an expressive Mustache API. The spec and the syntax, as they are today, did not block me.

I think we should not focus on the syntax. For me, it cannot be the right level, since there will always be some users asking for missing features, and the syntax will always be late.

Hooks are the right level. Handlebars, on this topic, is particularly brilliant.

[groue]

A simple example of the Handlebars expressivity : https://gist.github.com/1048968

[phloe]

Yeah - spec could bloat if features get added without merit... I like the idea of helpers/hooks.

You could pick and choose the helpers - and hopefully write/port them as you need them for other languages.

I like the syntax of Liquid's filters though (not too dissimilar to janl's helper syntax). Imagine if you wrote a generic "date" helper that could get a format argument like so:

{{myDate|date:"%Y %h"}}

Obviously myDate would be an ISO datestring :)

[ahem]

I really like the idea of helpers as well. I feel that lambdas introduce logic into the viewmodel, and IMO there isn't really any difference between logic in the template and logic in the viewmodel - neither of which are all that desirable.

The filters or modifiers from ctemplate or liquid, as @janl suggests above looks very interesting.

@groue's handlebars way might be a good solution as well, but is the best tag really {{#somthing }}? I mean #-sections already have way too much functionality in them (if-statements, loops, scoping and so on). Also, it would mask the helpername so that you could no longer access properties with that name, wouldn't it?

[groue]

@groue handlebars way might be a good solution as well, but is the best tag really {{#somthing }}? I mean #-sections already have way too much functionality in them (if-statements, loops, scoping and so on).

I don't know if Mustache should jump in the Handlebars train. Yet we should consider its design with interest. Many mustache/spec issues do not exist in Handlebars, because Handlebars is expressive enough to let developers find a solution to their problems without asking for a language patch. And this really should ring a bell for the Mustache designers.

Its key design decision was to push the rendering out of the "kernel", and put it in "userland", with built-in implementations for the most common cases (bools, loops, scoping, mainly). Since rendering is done in userland, Handlebars users can use the public Handlebars APIs in order to build their own extensions to the language. They are just as powerful as the built-in helpers, which use public APIs as well. This really was a smart move.

Mustache does not totally prevent this expressivity, but unfortunately the only hooks that the Mustache spec provides are lambdas, which are so weak they can not help implementing basic stuff like number formatting or array indices, for instance. As a consequence, because we live in a world that favors pragmatic solutions over theoretical purity, Mustache implementors give lambdas extra powers, out of the specification guidelines, or even provide other hooks, not even thought by the current specification (my GRMustache, maybe others, is in this case), or, worse, play with the syntax (and that's really not smart, since compatibility is key to Mustache).

My opinion is that we should externalize in userland some Mustache features as well, and really avoid touching the syntax too much. Even if sections gain even more responsabilities. Finding unambiguous and non-collapsing names for "active" sections and viewmodel properties should not be too hard.

Just for instance, GRMustache hooks let you do the following rendering, without any built-in support, whatsoever, for number formatting:

raw: {{float}}
{{#percent_format}}percent: {{float}}{{/percent_format}}
{{#decimal_format}}decimal: {{float}}{{/decimal_format}}

gives:

raw: 0.5
percent: 50 %
decimal: 0,5

Everything is done in userland, in the code of the GRMustache user.

In conclusion: first start with userland, check the expressivity of your API, check if you can do anything useful with it. And then, after the API has been carefully tested, that its expressivity has been proven, then maybe, think about providing a Mustache standard library of helpers/hooks, like ctemplate or liquid. Thinking about this standard lib right now is premature.

[ahem]

I agree that this should be done in userland. All the spec should do is provide the hooks to make it possible to provide this form of extensions. I still stand by my objection to adding more features to sections, however. There is already several open issues here regarding that (#14, #22, #23 and others) so I am not the only one who feels like this. Also there is this:

<time datetime="{{#format_isodate}}{{myDate}}{{/format_iso}}">{{#format_readable_date}}{{myDate}}{{/format_readable_date}}/time>

It may just me, but I do feel that this is more readable (even with the strange PHP formatting letters):

<time datetime="{{myDate|date:"%c"}}">{{myDate|date:"%j. %F %Y"}}</time>

The arguments in the proposed filter syntax seems to be somewhat of a necessity, if the hook should be useful for formatters. Writing a number of decimal formatting tags like {{#2_decimals_format}} {{#3_decimals_format}} {{#4_decimals_format}} and so on seems slightly silly to me.

[groue]

I personally feel that literal number and date formats (%j, %Y and friends), right in the template, are smelly: template localization is going to be painful, and not everybody speaks English. Literal formats are not a necessity, and should even be avoided.

I'd rather have indirect formats {{myDate|date:iso8601}}, {{myDate|date:short}} {{myDate|date:long}}.

And now I can't see why {{#format_iso8601date}}{{myDate}}{{/format_iso8601date}} is really less readable, even if it looks more verbose.

Moreover, sections allow for formatting not only a single number, but all numbers in a whole section of the template. Think: ISO date across a whole XML document, with a single section. Think: currency format across the whole invoice HTML document, again with a single section.

[bobthecow]

How 'bout

{{#date.iso8601}}{{myDate}}{{/date.iso8601}}
{{#format.iso8601date}}{{myDate}}{{/format.iso8601date}}

? Then you can have hashes full of formatter lambdas, and indirect formats rather than literal formats.

[phloe]

I'm sold on the indirect formats - much better than the hard-to-read php/ruby formating :)

[ahem]

Okay, so the PHP dateformatting example was a poor one, you're right in that literal date formats should not be used. But wouldn't implentation be a problem with the section based approach? Maybe this is too early to discuss actual implemention, but I do feel that it should at least be considered.

{{myDate|format_iso8601date}} is easy, it maps to something like registeredHooks.format_iso8601date(value);. This is easy to implement and easy to understand.

With sections it's possible that more formatters could be active at one time, so some kind of protocol is required to select which formatter will be used. In GRMustache you have selected a pretty simple protocol - only numbers are formatted so the formatters only have to be called if what is outputted is a number, and you only have to call the most recently enabled formatter.

If this were a more generic setup, where hooks could be used to format anything, then either each hook would need to provide a callback where it could opt in or out, as to whether it would like to format a given value, or you would need some other way to decide.

Maybe all formatters are called in the order they were enabled, starting with the most recently enabled one, and then they would have to return either the original value, or a formatted version of it.This would mean that a template like this: {{#format_numbers}}{{#format_dates}}{{myDate}}{{/format_dates}}{{/format_numbers}} would turn into: registeredHooks.format_numbers( registeredHooks.format_dates(value) ).

Either way, it is definitely more complex behavior, harder to implement and harder to understand... and I still feel that it is less readable :-)

[bobthecow]

The new "helpers" feature of Mustache.php (2.0-dev) makes this super awesome:

Given a set of formatters:

<?php

class DateFormatters {
    const SHORT_DATE = 'd/m/Y';

    public function __isset($key) {
        return method_exists($this, '_'.$key);
    }

    public function __get($key) {
        return array($this, '_'.$key);
    }

    public function iso8601($date) {
        return $this->parse($date)->format(DATE_ISO8601);
    }

    public function atom($date) {
        return $this->parse($date)->format(DATE_ATOM);
    }

    public function short($date) {
        return $this->parse($date)->format(self::SHORT_DATE);
    }

    private function parse($date) {
        return new \DateTime($date);
    }
}

Register them as helpers:

$m = new Mustache;
$m->addHelper('date', new DateFormatters);

And use 'em in all your templates!

{{# date.short }}{{ item.createdAt }}{{/ date.short }}

(note that this is fully supported by the current spec: it doesn't require anything besides lambda support and dropping a bunch of helpers in the top-level context stack frame)

[ahem]

@bobthecow It doesn't fully follow the current spec, does it? Is the dotsyntax in the spec now? Also, dropping helpers into the top-level context stack frame isn't either, i believe, but that is sort of what I think most of us here is getting at anyways.

How does Mustache.php solve the issue of multiple formatters active at one time? Or how would the name be rendered in a template like this:

{{# date.short }}{{ item.Name }} created at {{item.createdAt}}{{/ date.short }}

Would item.Name be passed to the date formatter? Would the "created at" string?

[bobthecow]

1. It's fully spec-compliant. dot notation is part of the spec as of v1.1.0 (March 4, 2011).
2. There's room for the PHP implementation of helpers in the spec: how you get things into your rendering context is of no concern to the spec. It's very language and implementation specific. In Ruby, you could add helpers by monkeypatching your Mustache class to add the formatter properties.
3. The whole thing would be passed to the formatter, just like any other lambda. Instead, you would do: {{ item.Name }} created at {{# date.short }}{{item.createdAt}}{{/ date.short }}

[ahem]

Ah, okay. This is different from what @groue have implemented in GRMustache then. If you can't put anything between the tags, then it doesn't make sense to have this behavior implemented with sections - and I still stand by that {{ item.createdAt|date.short }} is much easier to read.

Also rendering the value, and then parsing it again with the helpers is not exactly best case is it? I would expect that in some cases information would be lost before the helper function got a chance to format it.

[bobthecow]

I was just pointing out that it's possible to implement without extending the spec.

That said, the pipe style does look a lot cleaner. I could get on board with a "pipe notation" that is (mostly) syntactic sugar for chaining lambda callbacks, the same way "dot notation" is (mostly) syntactic sugar for chaining nested section tags.

Where these:

{{ foo.bar.baz }}
{{ foo | bar | baz }}

are both a shorthand for

{{# foo }}{{# bar }}{{ baz }}{{/ bar }}{{/ foo }}

but in the case of dot notation, it does the current context stack restrictions, and in the case of pipe notation, it passes the previous value to the next lambda.

This would save the to string and back trip, and would clean up templates, especially when running multiple filters.

[groue]

Ah, okay. This is different from what @groue have implemented in GRMustache then. If you can't put anything between the tags...

Indeed GRMustache's proposal can deal with {{#format}}...{{date}}..{{#foo}}...{{otherDate}}...{{/foo}}...{{/format}}.

And this can not be acheived with lambdas. Some other hooks had to be introduced.

While I was still stuck with the lambda frame, the most I could do was to implement something like @bobthecow, that is to say a lambda that render and use their immediate content.

@bobthecow, what about checking GRMustache API ? It's not funny to read a documentation, but at least there is a documentation: https://github.com/groue/GRMustache/blob/master/Guides/sample_code/number_formatting.md

It's fully spec-compliant.

It looks that your helpers render raw rendered strings (that is to say, strings that you trust will not be touched by the Mustache engine) : unfortunately, this is not a spec-compliant behavior.

What you may have missed, and no one could blame you, is that the spec requires lambdas to output a template string that will be parsed by Mustache engine, no the rendered string itself : https://github.com/mustache/spec/blob/master/specs/~lambdas.yml#L28

[bobthecow]

What you may have missed, and no one could blame you, is that the spec requires lambdas to output a template string that will be parsed by Mustache engine, no the rendered string itself.

Sorry, yeah, I realized that after I wrote the example :)

Mustache.php is spec-compliant, and it works just like you say. You would have to pre-render the string, so the body of parse would handle that in a full (not off-the-cuff and typed into a <textarea>) implementation :)

[bobthecow]

@groue I have seen the GRMustache API, and I have fundamental problems with it :)

I'd much prefer the syntax to use pragmas rather than sections when it's changing the meaning of {{ }} inline like that.

raw: {{float}}

{{%FORMAT percent}}
percent: {{float}}
{{%FORMAT default}}

{{%FORMAT decimal}}
decimal: {{float}}
{{%FORMAT default}}

[groue]

@bobthecow You're right. Actually, I really don't like that lambdas can not output anything that will be rendered raw. You always have to make sure your lambda won't output any {{, in case you would trigger unintentional Mustache formatting. This means that you can not output data that comes from untrusted source.

[groue]

I have seen the GRMustache API, and I have fundamental problems with it :)

I'd be happy to know them.

changing the meaning of {{ }}

???

[bobthecow]

???

It's saying "Between {{#THIS_TAG}} and {{/THIS_TAG}} treat all {{ of_these }} differently". That's inconsistent with the way sections work elsewhere in mustache. It feels more like a compiler directive to me, which is why I'd prefer pragma instead of section syntax.

[ahem]

The spec does specify that "Lambdas used for sections should receive the raw section string" (here), so I guess, if one were to implement a lambda that would treat 'variable blocks whose value is a number' different from the rest, one could just do that.

This isn't exactly how GRMustache implements it, but I believe the effect is similar. The formatterstacks could be considered helpers for building lambdas like this, I guess. @groue should of course correct me on this :-)

As I read the spec, however, the context object isn't supposed to be accessible from inside the lambda. The lambda is supposed to receive only the text inside it's own section unrendered, so it would be something like {{ number }}.

Without access to the context object, it isn't possible to format anything, and even with access to the context object, it still needs to implement it's own Mustache parser (or have some sort of deep knowledge of it's parent parser, so it can reach into it like GRMustache does).

IMO this is not a viable option - the goal should be that a user could provide a formatter like this (JavaScript):

{ decimal: function (x) { return typeof(x) === 'number' ? x.toFixed(2) : x; } }

and, with that attached somehow, expect it to format a template like this:

{{% decimal }}{{ pi }}{{/decimal}}

or this:

{{ pi|decimal }}

into the expected "3.14".

[spullara]

How about something that is simply rewriting the current syntax in a more terse format. For example, you might write:

{{#format_isodate}}{{myDate}}{{/format_iso}}

but allow it to be rewritten, with no change to the viewmodel, as:

{{#format_isodate myDate/}}

And have it literally do the same exact thing, just with a more compact syntax.

[groue]

Without access to the context object, it isn't possible to format anything, and even with access to the context object, it still needs to implement it's own Mustache parser (or have some sort of deep knowledge of it's parent parser, so it can reach into it like GRMustache does).

@ahem : I can not find the @pvande (spec writer) comment about this, but if I remember well, lambdas do not have access to context, but they can still render their "inner content" (have access to a function that renders their template string with the current context). @bobthecow lambdas are hence legit.

{ decimal: function (x) { return typeof(x) === 'number' ? x.toFixed(2) : x; } } (1) {{% decimal }}{{ pi }}{{/decimal}} (2) {{ pi|decimal }} (3) {{ decimal pi }} (@spullara proposal)

(1): I can't see how the x parameter would get the pi value, since this syntax allows to write {{%decimal}}...{{pi}}...{{e}}...{{/decimal}}, without any ovbious way to fill the x parameter.

(2) and (3): Why not, and let's admit that Mustache 2.0 already exists, and is Handlebars (since (3) is Handlebars syntax).

---

GRMustache splits its hooks in two sets: the regular lambda hooks, that can alter the template string, and the "delegate" hooks, that can alter the values about to be rendered.

Mustache lambda hooks say: "I'm about to render this template string. Should I render something else?".

GRMustache value hooks say : "In case you'd be interested, I'm about to render (value) in response to the "name" key. Should I render something else?"

I want to make it clear to anybody that those value hooks are a work in progress, not a formal proposal. I just wanted that my library users (including myself) can render their damned data, without writing too much boilerplate code, while maintaining compatibility (that is to say, with a clear frontier between regular Mustache, and GRMustache add-ons).

Handlebars is not a perfect solution, neither. Especially, the way they render array indices looks handy as long as the implementation language allows to extend any object with any key (https://gist.github.com/1048968) : with javascript, they can add their indices to the array items before they are rendered, so that the {{index}} tag renders them. Unfortunately, more rigid languages like Objective-C do not allow this very easily. GRMustache can render array indices without altering the viewmodel objects like Handlebars: https://github.com/groue/GRMustache/blob/master/Guides/sample_code/counters.md

[ahem]

(1): I can't see how the x parameter would get the pi value, since this syntax allows to write {{%decimal}}...{{pi}}...{{e}}...{{/decimal}}, without any ovbious way to fill the x parameter.

@groue That was why I used % instead of # in the tag. The idea is that %-blocks should signify something like the GRMustache hooks you have implemented. As you say, they work different from regular mustache hooks, so I really do feel they should have different syntax.

My proposal for hooks like that would be that instead of the type-based protocol you have implemented (NumberFormatters work for number, DateFormatters for dates and so on) every variable tag would just pass it's value through all active helpers before printing it. The helper functions are then responsible for returning the original value if they don't intend to alter it.

(2) {{ pi|decimal }} (3) {{ decimal pi }} (@spullara proposal)

I like these solutions for their simplicity. This is easy to understand, easy to implement and easy to use. They don't allow users to register global helpers like your implementation does, though, which may be a weakness.

Oh, and regarding the @bobthecow lambdas, they may be legal, but they still depend on parsing the default string representation of a value back into the original value before formatting, and that may not be possible in all cases. And it is is slightly weird behavior anyways, isn't it?

[groue]

@ahem Got it, thanks for the explanation and the feedback. Now I need to let these fresh ideas boil in my head :-)

[groue]

Oh, and regarding the @bobthecow lambdas, they may be legal, but they still depend on parsing the default string representation of a value back into the original value before formatting, and that may not be possible in all cases. And it is is slightly weird behavior anyways, isn't it?

@bebthecow's lambdas depends on PHP weak typing : "3.14" the string and 3.14 the number are the same. In other languages, we'd have to 1. render the inner content, 2. convert to number, and 3. format. That's pretty contrived, I agree with you.

I won't blame him, though. Mustache spec, as it is, forces us to hack around like that. I'm really happy we're having this discussion today, so that a clear light is shed on current limitations, and that today's hacks look like what they really are: hacks.

[pvande]

From a use-pattern perspective, the correct way to model this problem is by using a method (not a lambda), as @janl suggested.

class ViewModel < Mustache
  def initialize(price)
    @price = price
  end

  def msrp
    '$' + Math.round(@price, 2)
  end

  def global_msrp(currency)
    "#{Math.round(@price, 2)} #{currency}"
  end
end

  This thing costs {{msrp}} ({{#global_msrp}}USD{{/global_msrp}}).

Methods (again, not lambdas) have access to the context they were declared in, and so can usefully look up the data that's adjacent to them in the context stack. Methods can also receive the raw template string when used in a section. This lets you cleanly couple your formatting logic to the data you're presenting. This is the way Mustache expects you to handle this today -- your view model is deliberately more than just data, but also encompasses your presentation logic.

From an approachability perspective, the lack of helpers / formatters / data-driven functions is something that is often raised. I feel like I understand the desire, but remain unconvinced that it's the right decision for the language. I would happily change my mind if someone could show me a real-world(!) use case that Mustache cannot currently serve.

[spullara]

Because of the way scopes work in mustache it is pretty simple to put the helpers in the base scope and not pollute the view models at a lll.

Sam

On Wed, Mar 21, 2012 at 10:33 AM, Anders Hellerup Madsen reply@reply.github.com wrote:

@pvande From a technical perspective, I don't feel comfortable with helpers/formatters that are that are directly attached to the models. I think everyone can see the usefulness of, say, a site-wide date or currency formatter, instead of having to implement these formatting methods on every viewmodel class (which would lead to duplicated and hard to maintain code).

The methods could be added to some form baseviewmodel or, if the language supports it, use extensionmethods or mixins or something, but this leads to issues with serialization, and I really do feel that it complicates matters in an unnecessary way.

I think all of this boils down to that separting concerns between template, viewmodel and whatever generates the viewmodel, is no simple matter. I feel that the viewmodel should really just be data, and that the template should be responsible for presenting this data. You clearly feel differently.

My thoughts on this is that both the viewmodel and the template should be completely free from code, and whatever logic is needed while rendering the template (loops, if/else, formatting and simple transformations of data) should either be a feature of the template language or kept, as simple as possible, in a single, shared, place.

From an approachability perspective, the lack of helpers / formatters / data-driven functions is something that is often raised. I feel like I understand the desire, but remain unconvinced that it's the right decision for the language. I would happily change my mind if someone could show me a real-world(!) use case that Mustache cannot currently serve.

I don't know what real world examples to show if, say, a way to format dates or currencies or whatever, shared between several templates and viewmodels, isn't enough to convince you.

It seems like the demand for a helper feature in Mustache is there, though - not just from me, but based on the fact that Handlebars exists, and that several Mustache implementations already have added this one way or another. Also, @janl may suggest using methods, but he also writes in his Mustache 2.0 document that "Handlebars implements helpers. They are a good idea and we should outright steal them :)".

---

Reply to this email directly or view it on GitHub: https://github.com/mustache/spec/issues/41#issuecomment-4622593

[ahem]

@pvande From a technical perspective, I don't feel comfortable with helpers/formatters that are directly attached to the models. I think everyone can see the usefulness of, say, a site-wide date or currency formatter, instead of having to implement these formatting methods on every viewmodel class (which would lead to duplicated and hard to maintain code).

I think all of this boils down to that separting concerns between template, viewmodel and whatever generates the viewmodel, is no simple matter. I feel that the viewmodel should really just be data, and that the template should be responsible for presenting this data. You clearly feel differently.

My thoughts on this is that both the viewmodel and the template should be completely free from code, and whatever logic is needed while rendering the template (loops, if/else, formatting and simple transformations of data) should either be a feature of the template language or kept, as simple as possible, in a single, shared, place.

From an approachability perspective, the lack of helpers / formatters / data-driven functions is something that is often raised. I feel like I understand the desire, but remain unconvinced that it's the right decision for the language. I would happily change my mind if someone could show me a real-world(!) use case that Mustache cannot currently serve.

I don't know what real world examples to show if, say, a way to format dates or currencies or whatever, shared between several templates and viewmodels, isn't enough to convince you.

It seems like the demand for a helper feature in Mustache is there, though - not just from me, but based on the fact that Handlebars exists, and that several Mustache implementations already have added this one way or another. Also, @janl may suggest using methods, but he also writes in his Mustache 2.0 document that "Handlebars implements helpers. They are a good idea and we should outright steal them :)".

To @spullara

Because of the way scopes work in mustache it is pretty simple to put the helpers in the base scope and not pollute the view models at a lll.

@spullara That depends on the language. I had something about inheritance and extensionmethods in the first verison of my comment, but I removed it because it isn't true. I apologize for messing up the order of post...

But anyway, a helper such as the one @pvande demonstrates in his comment cannot be shared in any way. It is hardcoded to only work for the "ViewModel" viewmodel, and it only works for the "msrp" property. If I would like to add another currency value to the viewmodel, I would need to add another, similar formatter for that field, and if I would like to use this formatter on some other viewmodel I would have to do it all again. I think that is duplicated and hard to maintain code.

I don't think there is any easy way to pass an argument to the formatting function, that could be used to select the property to format, except writing it in the text inside the section, and IMO that isn't very pretty, and again, it isn't easy or even possible to select properties based on string representations of propertynames in all languages.

[groue]

@pvande : 1- How can we relieve the burden of writing dozens of stub methods/properties when there are many values to format?

2- I don't get how the Mustache subclass can format nested values, such as person.pet.price, for instance, in a template such as {{#people}}{{#pets}}{{price}}{{/pets}}{{/people}}. I'm curious about how it should be written. I'd be happy seeing some actual code.

Last point: not all Mustache implementations use a "viewmodel". Some render actual raw models. This is especially the case when the host language makes it painful (read, not practical) to create a specific class for each rendering. This is also the case when we render a data hierarchy. Hence the prevalent topic, in this thread, of providing "pollution-less" ways to achieve some features. Hence, also, my question number 2.

[davidsantiago]

I would say a real-world use case is the one that Shopify apparently had when they wrote Liquid: They wanted their users to be able to customize the appearance of their section of the site. There's no way you could give adequate control over the style without being able to make stylistic choices about the data inserted into the template, from the template. Most of the ideas being discussed here don't seem to include room for arguments to the helper functions, which would be a shame.

[pvande]

@ahem The role of the viewmodel is to house the data and logic being dispayed. This serves to enforce a separation between display logic (conditionals, iteration) and serialization / presentation logic. While other templating languages simply consume data, Mustache chooses to be deliberately different on this point.

This isn't for the sake of being contrary: your templates can be analyzed and optimized, your presentation logic is localized to the viewmodel (as opposed to across dozens of non-code files), and the viewmodel is an artifact you can test.

That's not to say that your viewmodel must have behavior. In the simplest cases, a hash more than suffices for your viewmodel, and most Mustache implementations handle this perfectly. In fact, the referential transparency of Mustache names means that in many cases, you could replace a "rich" viewmodel with a precomputed hash and notice no difference. (This is no longer true if your viewmodel mutates state, or if you use methods for sections.)

It's tempting to argue that creating rich behavior-full viewmodel objects, then, is a waste of time -- after all, if you can get the same results by simply building up a primitive data structure, what's the draw? The same argument applies to writing documentation or tests, but many of us have accepted that the time spent writing those things is valuable, even though it may double or triple the repository size without adding new functionality. Mustache promotes viewmodels over data objects for the same reasons: clarity, maintainability, and accountability.

As for the issue of keeping the transformations localized, most languages have mechanisms for doing just that. Static utility classes, reusable code modules, and first-class functions can all be used to effectively distribute behavior across a large number of objects. When viewmodels are code, reusability is a code-facing problem, with standard solutions.

---

Helpers are tricky. Their primary purpose is to push logic down into the template, which Mustache abhors. At the same time, it's the tool that's most familiar to most developers already, and it's frequently harder to transition from a primitive data object to a rich viewmodel than it is to argue for helper functions. Even if every Mustache user were writing expressive viewmodels, we'd likely still be having this discussion because of the strong temptation to push formatting behavior down the point of use. Mustache believes that viewmodels -- not templates -- are the point of use.

---

@groue To address your concerns:

1) Metaprogramming can go a long way towards that end.

2) Each person's pet should be a viewmodel capable of formatting the price appropriately.

3) If a Mustache implementation is capable of invoking methods, looking up properties, and fetching from hashes by key, then it uses a viewmodel. (If it isn't, it's not a Mustache implementation.) That's not to say that every Mustache implementation has a base class to inherit from, or even necessarily calls it out in the documentation. A viewmodel is simply an object graph that is capable of presenting itself, and that accurately describes the "data" argument consumed by every Mustache implementation.

---

@davidsantiago That's certainly an interesting case. I may need to think on that for a while...

[ahem]

@pvande This is an interesting dicussion. I feel that helpers are a necessary tool precisely because it would increase maintainability without adding logic to the template - two of the points you make against it.

your templates can be analyzed and optimized, your presentation logic is localized to the viewmodel (as opposed to across dozens of non-code files), and the viewmodel is an artifact you can test.

I think everyone will agree that putting actual codeblocks into templates (if that is what you mean by having untestable logic across dozens of non-code files) is a really bad idea. Fortunately no-one is suggesting that.

I don't feel that this:

var userViewmodel = {
    name: "Billy Bob",
    birthday: new Date('1981-12-02'),
    birthday_isoformat: function () { return isoformat(this.birthday);  },
    birthday_readableFormat: function () { return readableFormat(this.birthday); }
}
var postViewmodel = {
    title: "A great post",
    content: 'Lorem ipsum....",
    publicationDate: new Date('2012-03-21'),
    publicationDate_isoformat: function () { return isoformat(this.publicationDate); },
    publicationDate_readableFormat: function () { return readableFormat(this.publicationDate); }
}

var template = "{{# user }}{{ birthday_readableFormat }}{{ /user }}";

is any more maintainable or testable than this:

var userViewmodel = {
    name: "Billy Bob",
    birthday: new Date('1981-12-02'),
}
var postViewmodel = {
    title: "A great post",
    content: 'Lorem ipsum....",
    publicationDate: new Date('2012-03-21'),
}

var template = "{{# user }}{{ birthday|readableFormat }}{{ /user }}";

Note the two templates - my point here is that this does not add logic to the template, currently the template (or template writer, at least) still has to decide wether he wants to write bithday_readableFormat or birthday_isoformat. With helpers added the decision will become birthday|readableFormat or birthday|isoformat. In my opinion, no difference there - no added logic.

On the models however, the addition of helpers will remove the need to add the boilerplate dateformatting properties, and thus remove the need to test it. I see this as a good thing - the logic is in the readableFormat() and isoformat() functions, so they are the ones that should be tested.

Yes, in a dynamic language like JavaScript I could easily write some kind of adapterfunction like this:

function addDateformatters (obj, datename) {
    obj[datename + "_isoformat"] = function () { return isoformat(this[datename]); }
    obj[datename + "_readableFormat"] = function () { return readableFormat(this[datename]); }
}

addDateformater(postViewmodel, 'publicationDate');
addDateformater(userViewmodel, 'birthday');

but why should I have to? All of this is helpers implemented already! It's just boilerplate code that you have to write everytime you want to use it. I strongly feel that this is functionality the templatingsystem should provide, it should not be left as an exercise for the user to complete time and time again.

The other thing is, that this is really only that easy because JavaScript is an extremely dynamic language. In a language like C# or Java this might not be so easy, and then the boilerplate code will become a real problem. By adding a formalized hook to add this kind of helperfunctions, you would be able to simply the viewmodels and improve the maintainability of them - without adding logic to templates or sacrificing testability.

[groue]

Despite pvande's reluctance to acknowledge that not everybody writes trivial templates, and that users deserve consideration, I'd be happy working with all men of good will.

[bobthecow]

Since Real Artists Ship, here's where I'm leaning:

https://github.com/bobthecow/mustache.php/compare/feature/filters

The assumptions here are that:

1. Pipe notation is analogous to dot notation — it can be thought of as syntactic sugar for nested sections.
2. Filters are just variables in the normal context stack.
3. Values are not coerced into strings (or escaped) until they come out the other end of the pipe.
4. Unlike nested sections, the first value in the pipe is fetched prior to passing it to the next lambda.
5. If the first value in the pipe is falsey, or if any other value is not a lambda, the pipe evaluates to an empty string.

Also:

• Filters don't get parameters. That's logic in your template. See the commentary above regarding {{ foo | date.iso8601 }} vs {{ foo | date: "%j. %F %Y" }} (or whatever).
• Since this is a non-spec feature, it is toggled with a {{%FILTERS}} pragma tag.

In code:

<?php

$mustache = new Mustache_Engine;

$tpl = <<<TPL
{{%FILTERS}}
{{ greeting | case.lower }}, {{ planet | case.upper | !! }}
TPL;

$mustache->render($tpl, [
    'greeting' => 'Hello',
    'planet' => 'world',
    'case' => [
        'lower' => function($value) { return strtolower((string) $value); },
        'upper' => function($value) { return strtoupper((string) $value); },
    ],
    '!!' => function($value) { return $value . '!!'; }
]);

// hello, WORLD!!

Or (more likely if you're using Mustache.php's helper injection feature):

<?php

$mustache = new Mustache_Engine;
$mustache->addHelper('case', [
    'lower' => function($value) { return strtolower((string) $value); },
    'upper' => function($value) { return strtoupper((string) $value); },
]);
$mustache->addHelper('!!', function($value) { return $value . '!!'; });

$tpl = <<<TPL
{{%FILTERS}}
{{ greeting | case.lower }}, {{ planet | case.upper | !! }}
TPL;

$mustache->render($tpl, [
    'greeting' => 'Hello',
    'planet'   => 'world',
]);

[davidsantiago]

This is very close to what I was thinking as well. The one argument I would have is about not allowing arguments to the filters. Picking a date format is a great example of why you would want that feature. There's way more ways to format dates than you could reasonably want to write no-argument functions for. In fact you might write a small universe of these argument-free functions with names that were essentially replacing their arguments. If you want to call this logic in the template, I guess maybe so, but at least it's not application logic, reaching into a database or writing code to process the request in the template or anything like that. Not any more so than if you wrote a "filter" function that did those things and had no arguments. You could already write a rogue lambda function to do those things if you were so inclined (in fact, the spec on lambdas has a test that demonstrates how to write a lambda that actually updates application state: https://github.com/mustache/spec/blob/master/specs/~lambdas.yml#L56). Allowing filter arguments would make it easier to write compact code on the app side and easier to select the output you want on the template.

[pvande]

@bobthecow That's not too shabby; here are a few thoughts.

• Nested sections are always rendered outside-in, so this is not directly analogous. Then again, dotted names are not directly analogous to nested sections either (failed section lookups don't allow subsequent lookups to bubble).
• I'm curious about why a falsey initial lookup would preclude subsequent filters.
• What happens when the initial lookup is a lambda? Is the lambda poached for its value, or is the raw lambda passed through the filter chain? What happens when a filter returns a lambda - when it it called?
• I'm pretty sure that supporting object literals as filter properties would be a sign of madness, but I'm less certain that allowing additional lookup names as parameters is a disservice. Parameters certainly support a less stateful ViewModel, which seems desirable to me. It's certainly possible to do this while being non-evaling; I'm not sure what the difference between non-evaling and logic-free is in practice, though. Thoughts?
• It starts to get a bit more difficult to debug longer filter chains -- more keys to lookup means more chances to miss.
• How do these filter chains work in Sections? Inverted Sections?
• Nitpick: I'm assuming you mean that the first value in the pipe is fetched before being passed along, since it's not actually being interpolated into the template until after it's passed through the entire pipeline.

Thanks for investing the time in this. :)

[groue]

GRMustache has taken a similar route.

Actually, GRMustache has not extended the syntax of Mustache as much as {{planet | uppercase | !!}}. One still has to use sections in order to apply one's filters: {{#!!}}{{#uppercase}}{{planet}}{{/uppercase}}{{/!!}}.

However, despite this syntactic difference (which could be eluded with using | as some syntactic sugar), GRMustache filters provide with quite useful features as well on top of those seen above:

• since they apply to a full section, they can filter as many inner tags as they want: {{#uppercase}}{{greeting}} {{planet}}!{{/uppercase}} -> "HELLO MARS!".
• since they apply to full sections, they can be nested.
• they are provided with more information, and particularly the key that has provided the value, and the position of the tag in the template. This is wonderful for debugging. {{person.pet.name}} fails? GRMustache hooks know if the pet has no name, or if the person has no pet, or if no person was found at all, and where exactly this tag can be found.
• there is an implicit filter that applies to the full template rendering. This allows to extend Mustache features without any explicit markup.

In the documentation, you will find that all hooks are paired in methods named "willDoSomething", "didDoSomething" - this is required by the memory management in Objective-C (one has to be able to tear down what he has initially done). There are links to sample code as well.

Regarding a few @pvande questions :

I'm curious about why a falsey initial lookup would preclude subsequent filters.

Not the case in GRMustache filters

What happens when the initial lookup is a lambda?

1-arity lambdas are values just like others. But they have no meaning until executed, and filters can not execute them, lacking any useful context. So, basically, filters can not do nothing else but returning them untouched, or encapsulated in another lambda.

0-arity lambdas are evaluated before entering the filter.

It starts to get a bit more difficult to debug longer filter chains

Without any notion of "call stack", that's sure. It's the same difficulty as debugging deeply nested sections. I believe this subject could be left to nice implementors that would provide this information to the hooks.

How do these filter chains work in Sections? Inverted Sections?

Filters take values, and output values. Eventually the last value enters the Mustache rendering code, which processes it with classic rules.

Thanks for investing the time in this

You're welcome.

[pvande]

@groue Taking a look at your implementation, it looks like your lambdas take a Section object instead of a string ... interesting. That definitely takes us into the realm where it's impossible for us to validate things in a language agnostic way, but it does (rather conveniently) allow you to sidestep many difficult questions (e.g. should the lambda receive the raw template string or the rendered one?). It also gives you additional control over the output, the context stack, potentially data lookup... While an extended syntax for this may also be desirable, I think this approach might just solve most of the problems we've been looking at here, in addition to a few others that have nagged at me for a while.

So here are my questions for you:

• Is it important for implementations to have a consistent interface for such an object?
• Is there a clean way to extend this to lambdas used for interpolation?
• How might specifications for these interfaces be described in a language-agnostic way?

[bobthecow]

• Nested sections are always rendered outside-in, so this is not directly analogous. Then again, dotted names are not directly analogous to nested sections either (failed section lookups don't allow subsequent lookups to bubble).

True. It even points that out in the documentation on dot notation... That's why I said it "can be thought of" rather than saying "is" :)

Just like dot notation is a more handy approximation of nested sections, this is a more handy approximation of nested lambdas. E.g.:

{{# foo }}{{# bar }}{{ baz }}{{/ bar }}{{/ foo }}

~=

{{ baz | bar | foo }}

• I'm curious about why a falsey initial lookup would preclude subsequent filters.

It seemed to be consistent with the way falsey values are handled elsewhere. I guess it's inconsistent with the analogous nested sections though, as the {{# bar }} and {{# foo }} sections would indeed be passed an empty string — assuming the inverted inside-out resolution that we're glossing over for the sake of simplicity :)

That said, I'm not sold on short-circuiting the filter pipeline when it hits a falsey value. You'll see that I implemented it the other way first, but changed my mind when I started messing with date formatting... The naïve implementation in PHP would give you a formatted version of the unix epoch if you passed it a falsey value, which would then require the implementer to check for truthyness in every filter lambda. That said, I'm not sold either way, and I'd be more than happy to change mine around.

• What happens when the initial lookup is a lambda? Is the lambda poached for its value, or is the raw lambda passed through the filter chain? What happens when a filter returns a lambda - when it it called?

The initial lookup is resolved to a value (but not coerced into a string if it's not already). At each subsequent filter, it's assumed that the lambda will return something which should be passed directly to the next filter. Due to my answer to the question above, the chain is short-circuited in the event that any filter returns a falsey value, but like I said, I'm not sold on this bit.

• I'm pretty sure that supporting object literals as filter properties would be a sign of madness, but I'm less certain that allowing additional lookup names as parameters is a disservice. Parameters certainly support a less stateful ViewModel, which seems desirable to me. It's certainly possible to do this while being non-evaling; I'm not sure what the difference between non-evaling and logic-free is in practice, though. Thoughts?

My take on almost every argument I've heard for parameters: make a ViewModel. That's exactly where your statefulness and logic belongs. It's a companion class to go along with a template and encapsulate all the code that you're saving by removing it from your template. If you're into the whole idea of logic-free templates, really make 'em logic-free.

I'm willing to concede filters, because they're super-useful. But I think parameters are going too far.

• It starts to get a bit more difficult to debug longer filter chains -- more keys to lookup means more chances to miss.

True. Caveat emptor.

• How do these filter chains work in Sections? Inverted Sections?

Meaning how do they work in a tag like {{# foo | bar | baz }}?

I haven't implemented them there. In interpolation, I figured they're something of a special case for escaping. The value at the start of the chain is in some format, and eventually it needs to be a format fit for rendering as a string. This isn't required for sections (or inverted sections) since they don't need to format anything. They're just 0, 1 or many element lists.

• Nitpick: I'm assuming you mean that the first value in the pipe is fetched before being passed along, since it's not actually being interpolated into the template until after it's passed through the entire pipeline.

Yes. I meant fetched. I was trying to post that on a really sketchy airport wifi connection and the form kept timing out. I'll retcon that into what I meant, not what I said :)

[groue]

@groue Taking a look at your implementation, it looks like your lambdas take a Section object instead of a string ... interesting. That definitely takes us into the realm where it's impossible for us to validate things in a language agnostic way, but it does (rather conveniently) allow you to sidestep many difficult questions (e.g. should the lambda receive the raw template string or the rendered one?). It also gives you additional control over the output, the context stack, potentially data lookup... While an extended syntax for this may also be desirable, I think this approach might just solve most of the problems we've been looking at here, in addition to a few others that have nagged at me for a while.

So here are my questions for you:

Is it important for implementations to have a consistent interface for such an object?

@pvande, my opinion on this take is no, for four reasons:

• an implementation should be able to give the library user more information than required by the spec (debugging information, for instance).
• an implementation should be able to provide required information via language features. For instance, if a language support closures, there is the possibility that some information does not need to be provided as a function argument.
• an implementation should be able to implement underspecified behaviors, or optimistically specified behaviors. Mustache lambdas are in this category (how is the lambda supposed to be able to get the inner rendering of the section?).
• an implementation should be able to work around specification bugs. Mustache lambdas are in this category (because of their relation to the "set delimiter" tags, and their inability to avoid processing {{tags}} that would come from program input). If GRMustache allows for the specification behavior, it must explicitely chosen by the user. Under this view, GRMustache lambdas have two facets, and require custom versatile parameters.

The implementation should only tell what information it requires the library user to be provided with. Complying implementation would provide this information, in the form they want, and maybe more.

I admit that this ruins the embedding of PHP/Ruby/etc in the specification tests. The embedding of C/C++/compiled/etc was compromised from the start, anyway.

Is there a clean way to extend this to lambdas used for interpolation? How might specifications for these interfaces be described in a language-agnostic way?

Just like the best specifications out there: with well-written words illustrated by some sample code in a clear, concise, and widely-known language. YAML falls short on this topic.

A few last words. It's funny that you ask about GRMustache lambdas on a thread about filters. The two concepts should be clearly distinguished:

• lambdas are the objects able to process the template fabric, able to add tags, modify the raw template text.
• filters (delegates in GRMustache vocabulary) are the objects able to process the view model values.

Although they both are "callable", they do not need the same kind of information to perform.

For instance GRMustache lambdas take a "section" object that mainly provides with the section innner string, and the ability to render the section in the current context. Filters get a value, the key providing this value, and debugging information.

[groue]

@bobthecow,

As previously said, my filters can apply on a full section (being able to filter all tags in a section), and there is an implicit (without markup) filter that applies to the full template (able to filter all tags in a template and its partials).

This has a huge advantage: it solves for good a recurrent request from Mustache users: "advanced" loop rendering.

With GRMustache filters, you can render {{#items}}{{index}}: {{name}}{{^last}}, {{/last}}{{/items}} into "1: ham, 2: oranges, 3: bread" (noticed the missing last comma?), without preparing the view model for the index and last keys. Sample code

I wish this feature would be considered as a breakthrough in a parameter-less Mustache syntax, and a huge service provided to the library user.

Of course, should Mustache take Handlebars path, with {{#each_with_index items}} parameterized sections, I would have more difficulties defending my take.