mustache / spec

The Mustache spec.
MIT License
364 stars 71 forks source link

Proposal: Mutable context objects #72

Closed samwyse closed 10 months ago

samwyse commented 11 years ago

I've just waded through issue 38: Template inheritance. I know I'm a couple of years late to that dance, but I suspect that one reason why the proposal hasn't been adopted is that there was never any consensus on what should be done; there were at least two implementations that behaved slightly differently. So rather than adding to an already lengthy thread, I'm going to propose something different that accomplished the same goal: Mutable context objects (MCO). (And I think that this will also satisfy the goals of issue 63: Inline partials.

MCOs use two "new-ish" bits of syntax.

{{<}} {{! pushes an MCO onto the context stack }}
{{$key}}value{{/key}} {{! adds a new key-value pair to it }}
{{/}} {{! pops the MCO off of the context stack}}

While the mutable context object is on the stack, the rest of Mustache works the same as always.

{{<}}
{{$hero}}Mahatma Gandhi{{/hero}}
{{hero}} is someone I admire.
{{/}}

The preceding renders as:

Mahatma Gandhi is someone I admire.

Note that using the same key overwrites the previous value.

{{<}}
{{$hero}}Mother Teresa{{/hero}}
{{hero}} is someone I admire.
{{$hero}}Harriet Tubman{{/hero}}
{{hero}} is someone I admire.
{{/}}

The preceding renders as:

Mother Teresa is someone I admire.
Harriet Tubman is someone I admire.

Note how this interacts with partials. Here is an example of a partial:

{{! an example template }}
<html><head><title>{{title}}{{^title}}Default title{{/title}}</title></head>
<body>
{{heading}}
Hello, world!
{{footing}}
</body></html>

Now, let's see what happens when it is used like this:

{{<}}
{{$title}}Custom title{{/title}}
{{$footing}}That's all, folks!{{/footing}}
{{> template }}
{{/}}

You wind up with this:

<html><head><title>Custom title</title></head>
<body>
Hello, world!
That's all, folks!
</body></html>

Needless to say, you can push multiple MCOs onto the context stack; any new values are written to the most recently created MCO (and obviously disappear when it gets popped). Any objects added to the context stack after the MCO are considered immutable, and are skipped over when adding a value. Also, by the judicious use of {{#key}} and {{^key}} sections, you have complete control over how substitutions are performed.

BTW, I expect that the last example is going to be a common use case, so I'd suggest that the following should be interpreted as doing the same thing:

{{< template }}
{{$title}}Custom title{{/title}}
{{$footing}}That's all, folks!{{/footing}}
{{/ template }}

Finally, I know that all of this can be done via lambdas. I would note that (a) just because anything can be computed via a Turing machine doesn't mean that you should ignore other ways of doing it, and (b) lambdas are an optional part of the spec, so IMHO that isn't a good reason to reject this or any other proposal. Also, I'm fully aware that I'm re-using sigils that other implementations have used for years. Feel free to suggest others that I could use instead, but note that this spec could, like lambdas, be an optional part of the overall spec. Conflicting implementations could then ignore it, possibly later adding an option to their engine to toggle the interpretation.

Your thoughts?

samwyse commented 11 years ago

Here is bobthecow's example from https://github.com/bobthecow/mustache.php/pull/130, reimplemented using a mutable context object. First, the layout.

<!-- layout.mustache -->
<!DOCTYPE html>
<html>
  <head>
    <title>{{ title }}{{^ title }}My Alsome Site{{/ title }}</title>
    {{ stylesheets }}{{^ stylesheets }}
      <link rel="stylesheet" href="/assets/css/default.css">
    {{/ stylesheets }}
  </head>
  <body>
    <header>
      {{ header }}{{^ header }}
    <h1>Welcome to My Alsome Site</h1>
      {{/ header }}
    </header>
    <section id="content">
      {{ content }}{{^ content }}
    <p>Hello, World!</p>
      {{/ content }}
    </section>
    {{ scripts }}{{^ scripts }}
      <script src="/assets/js/default.js"></script>
    {{/ scripts }}
  </body>
</html>

Next, the input template.

<!-- page.mustache -->
{{< layout.mustache }}

{{$ title }}My Alsome Page{{/ title }}

{{$ stylesheets }}
  <link rel="stylesheet" href="/assets/css/page.css">
{{/ stylesheets }}

{{$ header }}
  <h1>My page has some items!</h1>
{{/ header }}

{{$ content }}
  <ul>
    {{# items }}
      <li><a href="{{ link }}" title="{{ name }}">{{ name }}</a></li>
    {{/ items }}
  </ul>
{{/ content }}
{{/ layout.mustache }}

The only problem with doing it like this is that it would result in four blank lines before the "" line. The simplest solution would be to simply omit the blank lines from the template; another would be to change the page to use a special lambda that suppresses empty lines, i.e.

{{< layout.mustache }}{{# omit_empty_lines }}
....
{{/ omit_empty_lines }}{{< layout.mustache }}

A few observations: All occurrences of "{{$ name }}default value{{/ name }}" in the template have been replaced by "{{ name }}{{^ name }}default value{{/ name }}". In Unix shells, this is called Substitution with a default value. MCO syntax is more verbose, but allows greater flexibility.

Substitution for actual value

{{# name }}substituted value{{/ name }}

This allows an different substitution to be used if a value is set, for example, "{{# name }}The value of name is {{ name}}{{/ name }}{{^ name}}Name is unset.{{ name }}"

Substitution with a default assignment

{{^ name }}{{$ name }}default value{{/ name }}{{/ name }}{{ name }}  

This allows an undefined value to get a default that can then be used throughout the remainder of the template.

spullara commented 10 years ago

Do you have any idea how you might handle the append case?

samwyse commented 10 years ago

I'm not sure what you mean by "the append case". Since {{$ name }}foo{{/ name }} creates 'name' with value 'foo'. I'd say that {{$ name }}{{name}}bar{{/ name }} overwrites the previous value with 'foobar'. But I suspect that you really want to talk about lists, not string concatenation.

Since '$' is the sigil for scalars, obviously '@' should be the sigil for lists/arrays (at least to anyone knowing Perl or PHP). Unfortunately, that sigil is already used for something else, so we need to think of another. I'm going to assume '$$', which (sort of) implies multiple values. But there are things I don't like about it.

Obviously {{$$ name }}item1{{/ name }} {{$$ name }}item2{{/ name }} would create an list of two items. That's OK, but what about other things?

It's equally obvious (to me, anyway) that {{$ name }}{{/ name }} creates a falsey value for 'name'. If you agree, then by analogy {{$$ name }}{{/ name }} {{$$ name }}item2{{/ name }} should create a list of two items, the first of which is 'falsey'. That seems simple enough.

What about interactions between '$' and '$$'? What does {{$ name }}item1{{/ name }} {{$$ name }}item2{{/ name }} mean? How about {{$$ name }}item1{{/ name }} {{$ name }}item2{{/ name }}? And of course, {{$$ name }}item1{{/ name }} {{$ name }}item2{{/ name }} {{$$ name }}item3{{/ name }}

The preceding examples are why I don't like using '$$', by the way. How easy is it to even notice that while most of them are using '$$', a few are using '$'.

And what if any of those values are empty strings, as earlier?

I know what I think should happen, but I'm already forcing enough down peoples' throats as it is.

groue commented 10 years ago

FYI @samwyse:

I'm not sure what you mean by "the append case"

The append case is today only implemented by GRMustache, as far as I know. It allows a section to be overridden several times, contents being then concatenated.

Given layout.mustache:

Content: {{$content}}Default{{/content}}

This template would render Content: Default:

{{<layout}}{{/layout}}

This template would render Content: Overridden:

{{<layout}}
{{$content}}Overridden{{/content}}
{{/layout}}

This template triggers the concatenation: it would render Content: AB:

{{<layout}}
{{$content}}A{{/content}}
{{$content}}B{{/content}}
{{/layout}}

This works also through multiple inheritance:

special_layout.mustache:

{{<layout}}
{{$content}}special {{/content}}
{{/layout}}

This template would render Content: special stuff:

{{<special_layout}}
{{$content}}stuff{{/content}}
{{/special_layout}}

The rationale (see #38 ): the analogous mechanism in Ruby On Rail's is their yield/content_for pair. It does concatenate multiple content_for, and there aren't much questions on StackOverflow from people who want to disable the feature - so I guess they hit the sweet spot. This is why GRMustache went this way.

samwyse commented 10 years ago

I'm going to work through your examples, but let me say that this is one of my use cases. I believe that MCOs give you additional power to decide how things are done, for instance, if you want to prepend or append when you concatenate. I've reproduced your examples, using MCOs but not the 'syntactic sugar'; I feel this better shows exactly how they will work in these cases.

Given layout.mustache:

{{! If 'content' is truthy, use it, otherwise use 'Default'. }}
Content: {{content}}{{^content}}Default{{/content}}

The following template would render Content: Default:

{{<}}{{/}}{{>layout}}

The following template would render Content: Overridden:

{{<}}
{{$content}}Overridden{{/content}}
{{/}}{{>layout}}

This works also through multiple inheritance:

special_layout.mustache:

{{!
Note that '$' doesn't define 'content' until the closing sigil. This
means that the enclosed 'content' refers to an object higher in the
context stack.  This, in turn, means that we can control if we want
the previously defined value to be prepended or appended.
}}
{{<}}
{{$content}}special {{content}}{{/content}}
{{/}}{{>layout}}

This template would render Content: special stuff:

{{<}}
{{$content}}stuff{{/content}}
{{/}}{{>special_layout}}

I moved this example to the end, because I'm not sure what's going on. If 'content' is meant to be a list of values, then we'll need the extra stuff I talked about upthread (and the assumption that lists are rendered by concatenating all items). If we are just defining a value as a concatenation, perhaps that is built up slowly over several lines,then this should work.

This template uses a concatenation: it would render Content: AB:

{{<}}
{{! Initially, 'content' is undefined, i.e. falsey, so this is 'A'. }}
{{$content}}{{content}}A{{/content}}
{{! Now, 'content' is truthy, i.e. 'A', so this sets it to 'AB'. }}
{{$content}}{{content}}B{{/content}}
{{/}}{{>layout}}
groue commented 10 years ago

Syntax is arguably more complex, but it's interesting.

I have a question: your mutable contexts are filled with HTML, not text: {{<}}{{$content}}{{string}}{{/content}}{{/}} will, I believe, fill content with &amp;, given & for the key string. So shouldn't you use triple mustache tags when rendering this value? Wouldn't {{content}} be rendered &amp;amp;? Shouldn't you write {{{content}}} whenever your want accurate rendering of values set through mutable contexts?

Another question: what do you mean by the following sentence?

And I think that this will also satisfy the goals of issue 63: Inline partials.

samwyse commented 10 years ago

@groue commented:

Shouldn't you write {{{content}}} whenever your want accurate rendering of values set through mutable contexts?

Yes, you should. Another test case, I believe. Thanks.

Another question: what do you mean by the following sentence?

And I think that this will also satisfy the goals of issue 63: Inline partials.

59 was "Idea: named html chunks", wherein one could say this:

{{ *widgetName }}
  <div>widget code</div>
{{/}}

and later use it like this: {{ **widgetName }}

I'll just say the same result can be achieved using MCOs:

{{$ widgetName }}
  <div>widget code</div>
{{/ widgetName }}

which can later be used like this: {{ widgetName }}.

59 was closed in favor of #63, Inline partials, which was suppossedly the same thing. However, note that MCOs resolve embedded tags when they are first scanned. #59 was silent on this, but (if I'm reading it correctly) #63 expects that resolution should be delayed until they are used.

Consider this:

{{$ person }}Socrates{{/ person }}
{{$ statement }}{{ person }} is mortal{{/ statement }}
{{$ person }}Plato{{/ person }}
{{ statement }}

An MCO should resolve this as 'Socrates is mortal' (using the definition of 'person' at the time that 'statement' is defined), while inline partials would resove it as 'Plato is mortal' (using the definition of 'person' when 'statement' is resolved).

So, how to make MCOs behave like inline partials? By using alternate delimeters. Consider this:

{{$ person }}Socrates{{/ person }}
{{$ statement }}{{=[[ ]]=}}{{ person }} is mortal[[={{ }}=]]{{/ statement }}
{{$ person }}Plato{{/ person }}
{{ statement }}

I haven't mentioned it before (because it only just now occurred to me) but values in MCOs have a two-phase resolution process. In this case the first phase leaves you with '{{ person }} is mortal', and the second results in 'Plato is mortal'. And this is exactly what is proposed for inline partials.

samwyse commented 10 years ago

I've consolidated the preceding discussion into a formal spec for MCOs. It may be found here: https://github.com/samwyse/spec/blob/master/specs/~mutable.yml

samwyse commented 10 years ago

Oops! Hit the wrong button!

groue commented 10 years ago

The fact that you still do not use triple mustache for rendering the HTML chunks stored in your "MCOs" is, I must say, troubling. What about starting using &, < and > in all your examples, and see how they behave (since I believe you have a running implementation)?

On building on top of change delimiters tags: I strongly advise against it. Those {{=[[ ]]=}} tags prevent Mustache from having a grammar, and force developers to write a custom parser. They are a joke that, I hope, will be removed one day. Some have already well understood that. For example, handlebars.js has not change delimiters tag, a grammar (https://github.com/wycats/handlebars.js/blob/master/src/handlebars.yy), and users who need to render the sequence {{ use a backslash escape.

groue commented 10 years ago

Another point: delayed resolution is already a built-in feature of template inheritance:

layout.mustache:

{{# items }}
{{$ item }}
{{/ items }}

This template renders Banana, Ham, Coffee with appropriate input (fetching the names from the items collection used by the layout):

{{< layout }}
{{$ item }}{{ name }}, {{/ item }}
{{/ layout }}
samwyse commented 10 years ago

The fact that you still do not use triple mustache for rendering the HTML chunks stored in your "MCOs" is, I must say, troubling.

​Hey, I did use it in the "MCOs as Named HTML Chunks" test. And as these specs are a straight copy of the examples I've been using up until now, I wasn't thinking about adding new stuff to them. 

What about starting using &, < and > in all your examples, and see how they behave (since I believe you have a running implementation)?

​I don't have an implementation of MCIs yet. I'm one if those crazy guys who likes to write test cases first. But give me a few days… and then I'll go back and sprinkle more HTML entities into the examples. 

On building on top of change delimiters tags: I strongly advise against it. Those {{=[[ ]]=}} tags prevent Mustache from having a grammar, and force developers to write a custom parser. 

​Right now, I'm working to extend the existing spec, and not break existing implementations, so the change delimiters stay, at least for now.

For example, handlebars.js has not change delimiters tag, a grammar (https://github.com/wycats/handlebars.js/blob/master/src/handlebars.yy), and users who need to render the sequence {{ use a backslash escape.

​Hmmm, I'll admit I hadn't even thought about using yacc to parse Mustache. OTOH, I suspect that while Mustache won't fit the definition of formal grammars from my Comp. Sci. class, yacc could still parse it, you'd just need a custom lexical scanner; I don't think that lex could handle changing delimiters.

samwyse commented 10 years ago

Another point: delayed resolution is already a built-in feature of template inheritance:

​And also lambdas. But there are open issues to specifically disallow it in other places (which I think is a bad idea, personally), so I thought I'd better specifically allow it for MCOs.

I had given some thought about adding a quoting sigil (say, {{" stuff}}, as opposed to changing delimiters or using backslashes) to flag delayed resolution of tags, but I didn't want to add new syntax if at all possible and changing delimiters seems to get the job done. I must admit, however, that It would make that one example at lot easier to read.

groue commented 10 years ago

The fact that you still do not use triple mustache for rendering the HTML chunks stored in your "MCOs" is, I must say, troubling.

Hey, I did use it in the "MCOs as Named HTML Chunks" test. And as these specs are a straight copy of the examples I've been using up until now, I wasn't thinking about adding new stuff to them.

Yes, you do. I just mean that since mutable contexts can only store HTML, it's better to show it clearly by always using triple mustache when you render them. Using simple double mustache is almost always a bug, since one almost never wants double escaping.

I thus think that all your examples should use triple mustache when outputting the content of a mutable context, as this should be the most frequent way to use them, and as you don't want to make people (including you) be mislead, and misuse your mutable contexts.

I hope you will agree on that point.

If so, I expect your own use of triple mustache to help yourself get a better overview of your own proposal. This may change your mind about its syntactic qualities/shortcomings. It may also open your eyes on the fact that the default mustache way (double mustache) actually opens an opportunity for your feature for being misused, with a delayed bad surprise: double mustache for mutable context rendering won't bite until they are fed with &, <, or >.

On building on top of change delimiters tags: I strongly advise against it.

Right now, I'm working to extend the existing spec, and not break existing implementations, so the change delimiters stay, at least for now.

Sure. However I would not foster their use, for the reasons I have given above.

​Hmmm, I'll admit I hadn't even thought about using yacc to parse Mustache. [...] I don't think that lex could handle changing delimiters.

My point exactly. Lex can not handle them. Hence my critic against change delimiter tags, and my advice against relying on them for building higher-level features.

samwyse commented 10 years ago

Using simple double mustache is almost always a bug, since one almost never wants double escaping.

You are probably correct.

I would not foster [the use of change delimiter tags], for the reasons I have given above.

You are bringing me around to your point of view. I see two ways to go, depending on whether the default behavior should be early evaluation or late.

If the default is to evaluate early, then you need to quote things you want evaluated later.

desc: An MCO is scanned when it is defined.  All tags but one are

evaluated then. template: | {{<}} {{$ man }}Socrates{{/ man }} {{$ proposition }}{{ man }} is mortal{{/ proposition }} {{$ man }}Aristotle{{/ man }} {{ proposition }} {{/}} expected: "Socrates is mortal"

desc: An MCO is scanned a second time is when it is used.  A

double-quoted tag will delay evaluation until then. template: | {{<}} {{$ man }}Socrates{{/ man }} {{$ proposition }}{{" man }} is mortal{{/ proposition }} {{$ man }}Aristotle{{/ man }} {{ proposition }} {{/}} expected: "Aristotle is mortal"

If the default is to evaluate late, then you need to unquote things you want evaluated early. A back-quote sigil (stolen from LISP) will do that:

desc: An MCO is scanned when it is defined.  Only back-quoted tags are

evaluated in this pass. template: | {{<}} {{$ man }}Socrates{{/ man }} {{$ proposition }}{{` man }} is mortal{{/ proposition }} {{$ man }}Aristotle{{/ man }} {{ proposition }} {{/}} expected: "Socrates is mortal"

desc: An MCO is scanned a second time is when it is used.  All other

tags are evaluated then. template: | {{<}} {{$ man }}Socrates{{/ man }} {{$ proposition }}{{ man }} is mortal{{/ proposition }} {{$ man }}Aristotle{{/ man }} {{ proposition }} {{/}} expected: "Aristotle is mortal"

In either event, I think that either type of quotes can be composed with all other sigils. For example, '{{"# man }}defined{{/ man }}' and '{{"> partial}}' would force late evaluation of sections and partials. (And note that I did not prefix the '/' with a quote, quoting quotes the entire section.)

HTML entities are only evaluated during one of the passes, not both; I'm not sure it makes much difference which, but we can assume as late as possible if it does. Changing "mortal" to "mortal (& so must die)" in the above templates will cause the output to read "mortal (& so must die)", not "mortal (&&amp; so must die)".

groue commented 10 years ago

You've been a long way. Thanks for your dedication.

Quoting your initial post in this thread:

I've just waded through issue 38: Template inheritance. [...] Rather than adding to an already lengthy thread, I'm going to propose something different that accomplished the same goal.

You have worked enough on your topic so that you are now able to compare your proposal and the template inheritance as described in issue #38. No more wading through: you're able to put the same energy in understanding #38 as I put in following you in your mutable contexts, and pointing at their (sometimes unintended) consequences.

samwyse commented 10 years ago

You have worked enough on your topic so that you are now able to compare your proposal and the template inheritance as described in issue #38. No more wading through: you're able to put the same energy in understanding #38 as I put in following you in your mutable contexts, and pointing at their (sometimes unintended) consequences.

"Wading through" does involve a certain amount of effort. There are two things that I dislike about template inheritance, as described. I'm going to refer to your gist for examples, but I think that my complains hold true for everyone's different template proposals.

At first glance, the '$' sigil seems to be context sensitive. Inside a template block (see 'sub.mustache'), {{$title}}Profile of {{username}} | Twitter{{/title}} is used to set a value. Outside of a template block (see 'super.mustache'), {{$title}}Default title{{/title}} means to render the value of 'title' (if defined), otherwise use 'Default title'. I'll admit, it's likely that '$' isn't context sensitive, it may be doing the same thing in each instance, and template blocks don't render their content, they only apply the side-effects of the tags within. In other words, '$' means substitution with a default assignment, i.e. {{title}}{{^title}}Default title{{/title}} with the side effect of setting 'title' to whatever is rendered.

MCOs use '$' to set a value without rendering it. You can use the existing Mustache syntax to perform substitution with a default value, so I exclude adding the same meaning to '$'.

The '<' sigil also does two things at once. It defines a region where substitution with a default assignment can occur without rendering, and then it include a partial.

Since MCOs assign without rendering, we don't need a block that suppresses rendering, and we can use the existing '>' sigil to include a partial after performing assignments. That pretty much eliminates the need for the '<' sigil. I've kept it to define regions where MCOs are in effect, but after these discussions, I'm thinking that they can be eliminated altogether. In that case, an MCO is implicitly pushed onto the stack when rendering begins, and another is pushed whenever a partial is used.

I don't code Ruby, much less Ruby on Rails, so it's entirely possible that templates are understood by everyone in that community. However, I don't think that they are easily understood by users of other languages. Template syntax is certainly somewhat more concise, but conciseness doesn't seem to be a primary goal of Mustache.

To be honest, I had an ulterior motive in opening this issue: I wanted to see if anyone responded. Mustache seems to be moribund. Even uncontroversial proposals that appear to patch obvious flaws have been sitting open for months. There are a lot of implementations of Mustache, but they all extend and enhance the syntax in incompatible ways to patch the not-so-obvious flaws. A few implementations seem proud to have dropped certain parts of the spec that they dislike (handlebars.js has not change delimiters tag). There are several other changes that I could have suggested, but template inheritance had the most replies. I thought that a proposal to completely eliminate them would stir some debate. I am very glad that you joined my discussion, but I'm somewhat disappointed that you were the only person to do so.

Yes, I'm implementing my own version of the Mustache spec. The first thing I did was write a test harness that loads the entire spec and tests my implementation against it. I forked the spec so I could add in many pull requests that have been sitting idle, but also with a vague idea that if Mustache v1 is dead, I might re-purpose my version as Mustache v2. I don't know that I'll go that far, but my experiment has left me more confident to make my implementation Mustache-like instead of Mustache compatible.

groue commented 10 years ago

To be honest, I had an ulterior motive in opening this issue: I wanted to see if anyone responded. Mustache seems to be moribund. Even uncontroversial proposals that appear to patch obvious flaws have been sitting open for months. There are a lot of implementations of Mustache, but they all extend and enhance the syntax in incompatible ways to patch the not-so-obvious flaws. A few implementations seem proud to have dropped certain parts of the spec that they dislike (handlebars.js has not change delimiters tag). There are several other changes that I could have suggested, but template inheritance had the most replies. I thought that a proposal to completely eliminate them would stir some debate. I am very glad that you joined my discussion, but I'm somewhat disappointed that you were the only person to do so.

Yes, I'm implementing my own version of the Mustache spec. The first thing I did was write a test harness that loads the entire spec and tests my implementation against it. I forked the spec so I could add in many pull requests that have been sitting idle, but also with a vague idea that if Mustache v1 is dead, I might re-purpose my version as Mustache v2. I don't know that I'll go that far, but my experiment has left me more confident to make my implementation Mustache-like instead of Mustache compatible.

You are right. This repo hasn't evolved since a long time, and its owners don't even answer to any message (@defunkt & @pvande). I think they consider Mustache finished. I think that so does the public community. Mustache is now seen as a minimalist template engine, end of the story. A look at stack overflows reveals how little people are waiting from it.

My position is simple: I use the very Mustache engine I have written, and I don't want to regret my choice of using it. Since the language defined by the spec is not only minimalist, but a chore invented by a stubborn minimalist nazi, that punishes you very hard as soon as you start rendering a non-trivial document, I had to extend the spec until my lib has become fluid and extensible enough. I did it in the best directions I could find, for the user's sake. It looks like you are doing the same, and this is good.

This Mustache spec repository is now a forum. I've been happy discussing your feature with you on this forum :-)

jgonggrijp commented 10 months ago

Closing this, as the proposal was superseded by template inheritance actually making it into the spec (#125) and because the comments seem to have been going in a less than constructive direction.

If someone would like to give the idea of mutable context objects another chance, please feel welcome to create a new discussion for it.