text/template: no nice way to escape/suppress newlines

[eloff]

In many text/template scenarios, when creating text files (e.g. csv) for machine consumption, or text documents for human consumption, it's desirable to control the newlines output by the template. Currently this involves jumping through hoops by moving template tags around, making the template very unreadable. A simple e.g \ character at the end of a line would be nice to escape the newline so it doesn't end up in the output text, but the template tags can still be organized nicely in the template itself.

Seems there was already a request for this on go-nuts: https://groups.google.com/forum/#!topic/golang-nuts/H_d6P6av8nk

[adg]

However this might work, it would need to be backward-compatible with existing templates. That rules out \ at the end of a line.

[eloff]

Good point. How about nunjucks system of adding a - at the start and/or end of a block to eat leading/trailing whitespace? Nunjucks does that: http://mozilla.github.io/nunjucks/templating.html#whitespace-control

{{range .foos -}} // eats trailing whitespace
{{.}}
{{-end}} // eats leading whitespace (\n from previous line)

[adg]

Seems like a reasonable approach. Can you give me an example of where this might help, so that I can judge the feature in context?

[eloff]

Sure, the thread had an example of generating a tsv file I think, but my use case is generating program code. I end up with a lot of empty lines where I had control tags, range tags, if tags, etc.

so I have something that looks like:

{{with .BitShift}}
    {{if ne . 0}}val >>= {{ . }}{{end}}
{{end}}

{{with $mask := .BitMask}} {{if ne $mask 0 }}val &= 0x{{ printf "%x" $mask }}{{end}} {{if ne $mask 0 | and $.IsSigned }} val = val<<63 | val>>1 {{end}} {{end}}

Which produces a lot of empty lines. I have to put non-related things on the same line, make very long lines, etc to get the formatting I want. With some whitespace eating feature I could do:

{{with .BitShift-}} // eats the newline and the indentation on the next

line {{if ne . 0}}val >>= {{ . }}{{end}} {{end-}} {{with $mask := .BitMask-}} {{if ne $mask 0 }}val &= 0x{{ printf "%x" $mask }}{{end}} {{if ne $mask 0 | and $.IsSigned -}} val = val<<63 | val>>1 {{end-}} {{end-}}

Not sure if the dash character is the best choice, but something like that to suppress newlines and indentation before or after tags would do the trick.

On Mon, Feb 23, 2015 at 10:12 PM, Andrew Gerrand notifications@github.com wrote:

Seems like a reasonable approach. Can you give me an example of where this might help, so that I can judge the feature in context?

— Reply to this email directly or view it on GitHub https://github.com/golang/go/issues/9969#issuecomment-75690901.

[josharian]

I've also frequently run into this while generating code and been frustrated by it. go/format can take care of multiple newlines, but it never goes from one newline to zero.

[adg]

The current way to avoid a newline is to do this:

Some text without a {{/*
*/}}newline in the middle.

which is a little verbose for such a minor signal. It would be helpful to have some way to signal that we don't want a newline.

1. Do we want the signal to be part of existing actions? (As in, {{end FOO}} where FOO is some token that doesn't collide with the current syntax.) Or, should "no new line" be an action of its own? ({{FOO}} for example) The former has the benefit of brevity, the latter has the benefit of being unambiguous and is probably cleaner to implement
2. Regardless of the answer to question 1, what should FOO be? A dash (-) is one idea, but there are many other things we can use. Back slash (\) seems more natural to me.

Some text without a {{\}}
newline in the middle.

A list of items:
{{range .Items \}}
{{.}} {{end}}

Should produce (where Items is [1,2,3,4]):

Some text without a newline in the middle.

A list of items:
1 2 3 4 

Hey @robpike what do you think?

[minux]

Could we add some kind of pre-template configuration options? {{ option +line-continuation }} or something like that.

and then we can just use the most natural syntax for line continuation: line 1 \ line 2

It also preserves compatibility with existing templates.

we can also make the line continuation character (or character sequence) configurable.

[eloff]

Or it can be a method on template, e.g. template.New(...).WithLineEscapes().Parse(...)

On Tue, Feb 24, 2015 at 7:32 PM, Minux Ma notifications@github.com wrote:

Could we add some kind of pre-template configuration options? {{ option +line-continuation }} or something like that.

and then we can just use the most natural syntax for line continuation: line 1 \ line 2

It also preserves compatibility with existing templates.

we can also make the line continuation character (or character sequence) configurable.

— Reply to this email directly or view it on GitHub https://github.com/golang/go/issues/9969#issuecomment-75881798.

[minux]

I don't propose it to be a method on Template is because I think this configuration is better contained in the template file itself, so that the code that uses the template doesn't need to know whether the template uses that feature or not.

I will also seamlessly update existing code and templates. Just recompile the code with Go 1.5 and you can start using this feature in templates.

[dworld]

I expect it as a builtin function, for example.

now:

        {{ range .Route }}
            {{ if contains .Method "GET" }} server.Get("{{.Path}}", {{$prefix}}do{{.Name}}{{$suffix}})
            {{end}} {{ if contains .Method "POST" }} server.Post("{{.Path}}", {{$prefix}}do{{.Name}}{{$suffix}}) {{end}}  {{ end }}

use a builtin functin _EAT_NEXTNEWLINE:

        {{ range .Route }}
            {{ if contains .Method "GET" }} server.Get("{{.Path}}", {{$prefix}}do{{.Name}}{{$suffix}}) {{ EAT_NEXT_NEWLINE }} {{ end }} {{ end }}
            {{ if contains .Method "POST" }} server.Get("{{.Path}}", {{$prefix}}do{{.Name}}{{$suffix}}) {{ end }} {{ EAT_NEXT_NEWLINE }} 
        {{ end }}

[cespare]

I run into this almost every time I use */template. I just did today: I was generating big SQL prepared statements using text/template and I had to choose between making the template easy-to-read and making the generated statement easy-to-read.

[sayeed-anjum]

I am using consul-template to generate text config files using a Go template. The templates can get quite complicated with nested ranges and conditionals. Making output result readable leads to ugly and unreadable code. Needs to be fixed at the earliest.

[loa]

+1

[zeevallin]

I'm also generating configuration which at some point will need to be read by humans. I really like the idea of {{- end }} for leading whitespace and {{ end -}} for trailing. +1

[ydnar]

We’ve used strings.Replace with a continuation char:

var (
    src = `{{range .Items}} \
        Hello, \
        {{.Name}} \
    {{end}}
    `
    t = template.Must(template.New("go").Parse(strings.Replace(src, "\\\n", "", -1)))
)

https://github.com/zonedb/zonedb/blob/master/build/generate.go#L85

[joncalhoun]

I agree that something like this would be useful. My biggest complaint with templates thus far is that they are hard to make human-readable without sacrificing the quality of the output.

That said, I suspect that there may be a lot of unique cases. Eg in my case I only wanted to trim newlines, so I used the following:

leading := regexp.MustCompile("(\n)*[{]{2}[-][ ]*")
tempBytes = leading.ReplaceAll(tempBytes, []byte("{{"))
trailing := regexp.MustCompile("[ ]*[-][}]{2}(\n)*")
tempBytes = trailing.ReplaceAll(tempBytes, []byte("}}"))

If you do address this, it would be nice to have a little flexibility but I don't really know how far you can go without it snowballing into too much.

[kaelumania]

+1

[gyuho]

+1, this might help us write more readable code:

Let's say we want to create this text:

CREATE TABLE IF NOT EXISTS my.table  (
    key VARCHAR(100) PRIMARY KEY NOT NULL,
    value1 INTEGER,
    value2 INTEGER
);

I had to:

package main

import (
    "bytes"
    "fmt"
    "html/template"
    "log"
)

func main() {
    queryStruct := struct {
        SchemaName string
        TableName  string
        Slice      []map[string]string
        LastIndex  int
    }{
        "my",
        "table",
        []map[string]string{
            map[string]string{"key": "VARCHAR(100) PRIMARY KEY NOT NULL"},
            map[string]string{"value1": "INTEGER"},
            map[string]string{"value2": "INTEGER"},
        },
        2,
    }
    tb := new(bytes.Buffer)
    if err := template.Must(template.New("tmpl").Parse(queryTmpl)).Execute(tb, queryStruct); err != nil {
        log.Fatal(err)
    }
    fmt.Println(tb.String())
}

var queryTmpl = `CREATE TABLE IF NOT EXISTS {{.SchemaName}}.{{.TableName}}  ({{$lastIndex := .LastIndex}}
{{range $index, $valueMap := .Slice}}{{if ne $lastIndex $index}}{{range $key, $value := $valueMap}} {{$key}} {{$value}},{{end}}
{{else}}{{range $key, $value := $valueMap}} {{$key}} {{$value}}{{end}}
{{end}}{{end}});`

http://play.golang.org/p/gl5CJWVry7

Thanks,

[progrium]

Just as a reference, Python Jinja's whitespace control is done with an optional global configuration to trim whitespace from lines with only a template block, as well as fine grained control based on the - pattern mentioned earlier.

http://jinja.pocoo.org/docs/dev/templates/#whitespace-control

The - pattern is also present in Ruby's ERB and others. If I were to prefer a syntax, it would be the one used everywhere else.

[mgcrea]

:+1: for {{- end }}-like syntax, to me this is really a must have.

[progrium]

{{+ would be rad to include the previous line whitespace for every line output of the block. so pipeline expression output doesn't have to care about indentation when indentation is important for the outer file (like yaml)

[josharian]

@robpike any thoughts about this? I just bumped into it again. I may be a princess, but this is definitely a pea.

[steve-jansen]

+1 for {{-end}} syntax, already familiar with this from Ruby/ERB

[azihsoyn]

:+1: +1 for {{-end}}

[ake-persson]

Lines that only has code statements should probably not generate a newline at all. It is nice to keep them on a separate line for readability.

Input: [1,2,3]

{{range .}}
Apple: {{.}}
{{end}}

Result:

Apple: 1

Apple: 2

Apple: 3

As for the other part {{-end}} seems similar to what Jinja does. So why not.

[jasonwbarnett]

+1. Both Ruby ERB and Jinja do somewhat similar things and I think it's an absolute requirement to have something similar in Go. Otherwise templating output looks stupid. Almost as stupid as this comment I'm leaving.

[dverbeek84]

+1 for ERB / Jinja style

[obieq]

+1 for some sort of resolution, preferably {{-end}}

[chasebolt]

+1 for {{- end }} syntax, matches well with ERB templates

[andrewwebber]

+1 for {{- end }}

[n-rodriguez]

+1 for ERB / Jinja style

[JakeWarner]

+1 for the suggested "{{- end}}" as well.

[gopherbot]

CL https://golang.org/cl/14391 mentions this issue.

[ake-persson]

Thanks!

[loa]

Thanks!

[zeevallin]

Wohoo, thanks!

[cgcgbcbc]

when will next release come？

[ianlancetaylor]

The next release is scheduled for February 1, 2016.

[brandoncole]

This will be fantastic to have!

[mopfeffe]

This comment is too late..but I've only recently joined the word of go templating. The discussion above is a good one...Perhaps Would another possible solution be to surpress blank lines by default if there is no output from the template instructions on that line?

Example:

Text Tex2 {{range $key, $pairs := ...some tree}} Text3 $key

{{end}}

The lines containing {{range.}} and {{end}} produce no actual output - so supress the blank line.

[davecheney]

This has been implemented, please see the go 1.6 release notes.

[thomasleveil]

and the link to the Go 1.6 release notes

First, it is now possible to trim spaces around template actions, which can make template definitions more readable. A minus sign at the beginning of an action says to trim space before the action, and a minus sign at the end of an action says to trim space after the action. For example, the template

{{23 -}}
   <
{{- 45}}

formats as 23<45.

[bradfitz]

@dsanthosh, you're commenting on a closed issue. We only use the issue tracker for bug reports, and don't track things once closed.

For questions about Go, see https://golang.org/wiki/Questions.