Allow multiple sequential delimiter token definitions for more readable template files

[eugenesvk]

Is your feature request related to a problem? Please describe.

Chezmoi has this awesome (and rare) feature that allows you to redefine template delimiter tokens with left-delimiter and right-delimiter settings at the top of any template file, so I have 2 modifications that solve 2 template issues:

1. language highlighting gets broken with {{}} as these are invalid language tokens in many languages, so I add a syntax-specific comment symbol before, so #{{}} in python doesn't trip syntax highlighting and is read as just a comment
2. the default {{ double brace type }} is something I don't like, so I use the smaller and nicer ‹›, the symbols that don't conflict with anything regular languages use, so you don't need to double or otherwise escape

But then there are still two unresolved issues that I was hoping this proposal (if implemented) would resolve

1. The comment syntax doesn't work for inline comments that would also require adjusting the right-delimiter token to include the end-of-comment string, and adding that would require adding it everywhere where it's not needed, which leads to ...
2. The extra language-specific comment symbols are not always needed, e.g., if you add a template var in a string, you don't need to add the # comment prefix, it's already parsed as a string and not trips any syntax highlighting rules

Describe the solution you'd like

Allow multiple rules of defining the delimiter tokens that would be executed sequentially, so that I could have, e.g.: Let's assume this is a language with // regular comment symbol and a /**/ inline comment symbol

# chezmoi:template:left-delimiter="//‹" right-delimiter="›"
# 1 ↑ replaces all uses of template vars/functions within non-string standalone language syntax
# chezmoi:template:left-delimiter="/*‹" right-delimiter="›*/"
# 2 ↑ replaces all uses within non-string inline language syntax (with an extra closing comment)
# chezmoi:template:left-delimiter="‹" right-delimiter="›"
# 3 ↑ replaces all uses within strings where an extra comment symbol is not needed
# should run last not to interfere with the first two rules

This would cover all the issues and allow me to use the best syntax for the job :) to make templates more readable

Describe alternatives you've considered

Not sure, don't think you can recursively feed a template with varying delimiter tokens

Additional context

For example, this json

  {"keys":["//‹$a›+backspace"], "command":"..."},
  {"keys":["//‹$a›+delete"   ], "command":"..."},
  {"keys":["//‹$w›+backspace"], "command":"..."},
  {"keys":["//‹$w›+delete"   ], "command":"..."},

would become

  {"keys":["‹$a›+backspace"],   "command":"..."},
  {"keys":["‹$a›+delete"   ],   "command":"..."},
  {"keys":["‹$w›+backspace"],   "command":"..."},
  {"keys":["‹$w›+delete"   ],   "command":"..."},

(I mean, ideally it should be

  {"keys":["‹$⎇›+backspace"],   "command":"..."},
  {"keys":["‹$⎇›+delete"   ],   "command":"..."},
  {"keys":["‹$◆›+backspace"],   "command":"..."},
  {"keys":["‹$◆›+delete"   ],   "command":"..."},

but chezmoi doesn't (yet :)) accept extra symbols in variable names)

[halostatue]

I don’t think that this is possible using Go’s text/template. As you noted, it would require the recursive application of templates, and the delimiter detection code is … simple. That is, the delimiters are matched using a regular expression for a single line, and they can’t be changed during execution (a limitation of text/template).

Your desire to use ⎇ and ◆ are restricted by the Go language specification. Those aren’t letters, so they can’t be used. You could use ƛ, Δ or even 東京, but they have to be letter or number forms.

[eugenesvk]

Thanks for a prompt response!

delimiters are matched using a regular expression for a single line,

That's a good idea, thought it was just a literal match, but regexes could work, no? E.g., (\/\/‹|\/\*‹|‹)|(›\*\/|›) could work non-recursively for the starting delimiter either on its own or prefixed with // or /* comment and for the closing delimiter either on its own or postfixed with */. Or are these too complex regexes for the template engine?

(and it's a pity Go wasn't unicode-forward-looking enough to have the possibility of these nice-looking templates)

[halostatue]

delimiters are matched using a regular expression for a single line,

That's a good idea, thought it was just a literal match, but regexes could work, no?

No, what I’m saying is that unlike mustache templates (where you can literally change delimiters mid-template), the delimiters must be set at template compile time, so chezmoi uses a regular expression (something like ^.*chezmoi:template:left-delimiter="([^"]+)"\s+right-delimiter="([^"]+)".*$) to find the delimiters and then sets those on the template before it gets executed.

One option — that I oppose, personally (even though I’m not fond of text/template) — would be to add an alternative templating engine for chezmoi. I suspect that with a modify_ template, one could do something like {{ . | toJson | output "alt-scripting-engine" "alt-scripting-engine-template.tmpl" }}, but that would require that the scripting engine have an executable that accepts its context variables over stdin.

(and it's a pity Go wasn't unicode-forward-looking enough to have the possibility of these nice-looking templates)

This is a choice similar to those made by Elixir and IIRC Ruby. They expect [\p{Letter}_][_\p{Letter}\p{Digit}]* for identifiers. What I couldn’t tell you is how either Go, Ruby, or Elixir react to identifiers that must be capitalized (exported for Go, modules for Elixir, and constants for Ruby) with languages that have no capitalization.

[twpayne]

Thanks for the proposal @eugenesvk and thank you for the comprehensive response @halostatue :)

As the requested feature is not currently supported by text/template, and I do not want to either fork text/template or add support for pluggable template engines (both too much work and maintenance effort for too little gain IMHO), I don't plan to implement this.