ES6-style template literals

[masak]

Things like this:

my foo = 42;
say(`The value truly is ${foo}, and no kidding!`);

I think it's a nice realistic case for something slang-like that calls back into the main language (more exactly, into the EXPR rule) in order to do the ${ } stuff. If we're able to do this nicely through some mechanism, we cover a lot of desirable ground.

[masak]

Here's my attempt to specify this as an is parsed macro:

constant string = /
    ([<!before "${"> <-[`]>]*)
    { make new Q::Literal::Str { value: str($0) } }
/;
constant interpolation = /
    "${" ~ "}" <EXPR>
    { make $<EXPR> }
/;

macro term:template(strings: Q::Literal::Str[], values: Q::Expr[])
    is parsed(/
        "`"
        $<strings> = <&string>
        [
            $<values> = <&interpolation>
            $<strings> = <&string>
        ]*
        "`"
    /) {

    my ast: Q::Expr = strings[0];
    for ^values.elems() -> i: Int {
        ast = quasi {
            {{{ast}}}               # what we already had so far
            ~ str({{{values[i]}}})  # the next interpolated value (stringified)
            ~ {{{strings[i + 1]}}}  # the next string part
        };
    }

    return ast;
}

Some comments:

• The way things work around make is still very murky and not clear in my mind. The main goal is that the regex parses the text and comes up with those Q::Literal::Str and Q::Expr nodes. Suggestions welcome.
• As I wrote this, I realized we need some way to call out to rules/regexes in the current scope, not just in 007's grammar. I went with <&foo> for now. Could go with some other syntax if needed.
• The loop came out really nice. Much better than I had expected.
  • Note how we incrementally build up the Qtree we want, building new nodes on the right all the time.
  • I first wrote the inside of the loop as mostly synthetic Q nodes, but then realized that this is much more easily expressed with a quasi. This is exactly what quasis are for.
  • The triple braces are a little bit jarring to the eyes, especially when there's so many of them. I could imagine this being a lot easier on the eyes if there were syntax highlighting. For some reason I picture the triple-brace characters as being colored #ff6600. :smile:

[vendethiel]

            ~ {{{strings[i + 1]}}}  # the next string part

should be

            ~ {{{strings[i]}}}  # the next string part

we want the string part in the current iteration, not the next one.

Looks good otherwise!

[masak]

I believe the implementation as it stands (with i + 1) is correct.

• The strings array will always be one element longer than the values array, due to the way the regex looks for first a string (though possibly empty), and then only value-string pairs.
• This is mirrored by the fact that ast starts out consuming strings[0], which covers the "base case" of a template string without interpolations.
• Therefore, whenever we consume a new value in values, the wanted index in strings will be ahead by one... from before the loop, and then by induction throughout.

What we're describing is a reduction on the zip of strings[1..*] and values, if that helps. Borrowing JS's Array.prototype.reduce for a while, and pretending that destructuring parameters already work in 007:

return zip(values, strings[1..*]).reduce(
    sub (ast, [v, s]) { quasi { {{{ast}}} ~ str({{{v}}}) ~ {{{s}}} } },
    strings[0]
);

[vendethiel]

oh, right. you use the same array name. Does that even work in perl 6?

[masak]

I'm a bit confused by the question. The same array name as what? As far as I can see, I'm not redeclaring anything in the code. (Though it is untested code, of course.)

[vendethiel]

        $<strings> = <&string> (first time here)
        [
            $<values> = <&interpolation>
            $<strings> = <&string> (second time here)

[masak]

Question now understood. Yes, that is a Perl 6 feature, and it feels like a good fit for what we need. Demonstrated here:

$ perl6 -e 'say ("o" ~~ / $<o> = "o" /)<o>.^name'
Match
$ perl6 -e 'say ("oo" ~~ / $<o> = "o" $<o> = "o" /)<o>.^name'
Array

Here's the relevant bit of S05:

If a subpattern is directly quantified with C<?>, it either produces
a single C<Match> object, or C<Nil>.  If a subpattern is directly
quantified using any other quantifier, it never produces a single
C<Match> object.  Instead, it produces a list of C<Match> objects
corresponding to the sequence of individual matches made by the
repeated subpattern.

And

If a subrule appears two (or more) times in any branch of a lexical
scope (i.e. twice within the same subpattern and alternation), or if
the subrule is list-quantified anywhere within a given scope (that is,
by any quantifier other than C<?>), then its corresponding hash entry
is always assigned an array of C<Match> objects rather than a single
C<Match> object.

[vendethiel]

Thanks!

[masak]

Hey, what do you know.

I did the for loop construct-me-a-chain-of-ops trick twice in a row, first in this issue and then in #176, without realizing they are related.

So here's a better implementation that doesn't re-invent the for-shaped wheel:

use metaop::reduce;

macro term:template(strings: Q::Literal::Str[], values: Q::Expr[])
    is parsed(/.../) { # as before

    sub stringify(v) { return quasi { str({{{v}}}) } }
    my parts = strings[0] :: zip(values.map(stringify), strings[1..*]);
    return quasi { [~]({{{parts @ Q::ArgumentList}}}) };
}

We've already said we might want to auto-cast Array of the appropriate values to things like Q::ArgumentList. If we decide not to do that, we'll need to wrap the parts expression in a manually constructed new Q::ArgumentList { arguments: ... }.