lazy evaluation of string templates

[gavinking]

The spec promises that string templates are lazily evaluated, but dodges the question of how exactly that works. This is actually a bit of a tricky problem if we want to continue to support the syntax:

String s = "Hello, " name "!";

Let me run through several options:

• Don't support the above syntax. A string template is a StringTemplate. If you want to get a String out of it, you have to call string explicitly.
• A string template is just a String. To re-evaluate it, you call interpolate() to get a new String. This doesn't work out if we continue to lower String to java.lang.String, but if we decide to change our minds about that (it is on the cards), then it could work.
• Make string interpolation eager all the time. To get lazy evaluation you write () "Hello, " name "!", or use some other weird syntax to get a String() instead of a String.

Thoughts?

[ikasiuk]

So the first option means that we would have to write:

String s = ("Hello, " name "!").string;

That's a little verbose but not totally horrible. Perhaps it would make sense to call .string implicitly when assigning a StringTemplate to a String. I guess that wouldn't be more "magic" than he miraculous lazy evaluation we currently promise. If we do that than most current code would work basically unchanged.

The second option has a big problem: it would effectively result in non-immutable Strings. And I'm pretty sure that would create more harm than good.

The third option is worth considering. It uses existing mechanisms and is more flexible than the first option because you have the possibility to use any function returning String. As with the first option, we could consider some additional magic: a string template could automatically be a function when assigned to a String() and otherwise it would always be a simple String. That would be completely compatible with existing code.

I think I prefer the third option (even without the magic).

[gavinking]

That's a little verbose but not totally horrible.

FTR, you would not need the parens.

Perhaps it would make sense to call .string implicitly when assigning a StringTemplate to a String.

That would be an implicit type conversion, so it runs into all the usual problems with implicit type conversions and type inference.

The second option has a big problem: it would effectively result in non-immutable Strings.

Not necessarily, reevaluate() could return a different instance.

[ikasiuk]

Not necessarily, reevaluate() could return a different instance.

That's not quite what I meant.

variable Integer x := 1;
String str = "" x "";
print(str);
x := 2;
print(str);

If I understand you correctly then that would be legal and print "1" and "2".

[gavinking]

If I understand you correctly then that would be legal and print "1" and "2".

That's not what I was really suggesting. I was thinking of:

variable Integer x := 1;
String str = "" x "";
print(str.interpolate());
x := 2;
print(str.interpolate());

[ikasiuk]

That's not what I was really suggesting.

I don't get it. Then what type does print accept?

[ikasiuk]

...or rather, what does interpolate return? Why can't I write print(str)?

[gavinking]

what does interpolate() return?

A String.

Why can't I write print(str)?

You can, but it won't re-evaluate the interpolated expression.

[ikasiuk]

Ah, so when a String object is created from a string template then the template is immediately interpolated, but the object internally stores both: the initially interpolated string and the original template. So using that String object directly will always return the same identical string, but interpolate returns an updated, newly interpolated version which may be different.

I don't like this very much because it's a bit confusing that a String object in some sense represents two different strings. So if some function receives a String then that may either be a "raw" string or a string with a template underneath. Should it use the string directly, or should it call interpolate first? An immutable string and a string template that can be evaluated into an immutable string are two different concepts. We shouldn't try to squeeze them into the same class.

[tombentley]

I like option 3, because it just seems to fit in so nicely with what we already have and is pretty intuitive.

[gavinking]

Well, the problem with option 3 is that it could be pretty ugly for writing UI templates in Ceylon.

[FroMage]

FTR, you would not need the parens.

So it'd be:

String s = "Hello, " name "!".string;

That's a bit confusing, no?

I like the idea that string templates are a different type.

[chochos]

If string templates were subclasses of String that would make things a lot easier, no?

[ikasiuk]

Well, the problem with option 3 is that it could be pretty ugly for writing UI templates in Ceylon.

Can you give an example?

If string templates were subclasses of String that would make things a lot easier, no?

That's basically option 2. As explained above, I don't think that's a good idea because a string template effectively represents a non-constant string.

[chochos]

But the string template itself will not change when turned into a string, only when invoking its interpolate() method

[ikasiuk]

But the string template itself will not change when turned into a string, only when invoking its interpolate() method

Yes, and I explained above where I see a problem with that.

[RossTate]

I vote for option 3. "Hello, " name "!" notation is gonna be one of the first things people learn; let's not make it complicated. Plus, I hate the idea of not being able to distinguish between a string value and a lazy string. Can you imagine making that the norm for integers? Also, option 3 highlights how simple closures are in Ceylon. It's super cool (to me) that () "Hello, " name "!" works and has (shorthand) type String().

[gavinking]

So, as discussed last week, due to the erasure of String, we think the only way to make this really work is to have separate String and StringTemplate types with no subtype relationship between the two of them.

String string = "Hello";
StringTemplate template = "Hello, " name "!";
String interpolated = "Hello, " name "!".string;

According to this definition, print("Hello, " name "!") still works, since print() calls .string on its argument. But you would have to write stuff like this:

shared actual String string => "Person[" firstName " " lastName "]".string;

Which is a little inconvenient, but I suppose we could get used to it.

The other option would be to make double-quoted strings do eager interpolation, and single-quoted strings do lazy interpolation, i.e.

String string = "Hello, " name "!".string;
StringTemplate template = 'Hello, ' name '!';

Unfortunately, there is already an existing distinction between single and double quotes: double quoted strings do automatic \-escape interpolation, which is really a totally orthogonal thing to this issue, AFAICT.

So I guess the choice would be between:

• requiring an explicit operation to interpolate expressions, i.e. "Hello, " name "!".string, or
• requiring an explicit operation to interpolate escapes, i.e. "Hello,\tWorld!".interpolated.

As @FroMage says: "Damn you, ASCII!".

Anyone for guillemots? ;-)

[ikasiuk]

The typical use case that was mentioned for lazy string templates is logging. So how would a logging function look with StringTemplates? It looks like constructing a non-trivial StringTemplate could be relatively expensive.

And if I have a function

void write(StringTemplate st) {...}

then a typical case might be that I have nothing to interpolate and just want to write a plain string:

write("Plain string");

But that doesn't work because a String is not a StringTemplate. So how would you write that correctly?

[gavinking]

The typical use case that was mentioned for lazy string templates is logging.

Actually, FTR, I don't see this as the typical use case. It's a use case, perhaps, but I'm more interested in what works for templating.

So how would you write that correctly?

Well that's the point of the discussion above. What we're saying is that "Plain string" would, in fact, be a plain String. We could even decide to use " vs ' to distinguish between immediate interpolate (String), and lazy interpolation (StringTemplate).

[ikasiuk]

So I guess the choice would be between:

• requiring an explicit operation to interpolate expressions, i.e. "Hello, " name "!".string, or
• requiring an explicit operation to interpolate escapes, i.e. "Hello,\tWorld!".interpolated.

But we also have character literals. So what about this:

String s = "Hello, " name "!";          // eager interpolation
StringTemplate st = 'Hello, ' name '!'; // lazy interpolation
String esc = "Hello," `\t` "World!";

It should be trivial to optimize the latter to a constant string at compile time.

[gavinking]

@ikasiuk honestly that would be perfectly fine by me.

(FTR, in the past I've proposed "Hello," ht "World!" as an alternative to backslash-escapes, but nobody went for it...)

[ikasiuk]

I don't know if I really need a StringTemplate class. A String() function as suggested by the third option in the original post seems to fulfill the purpose just fine.

[gavinking]

@ikasiuk perhaps, but the question about syntax stands. () "Hello, " name "!" is, to my way of thinking, unacceptably ugly in a Ceylon-based template language.

[gavinking]

P.S. The advantage of StringTemplate over String() is that print() does the right thing when passed a StringTemplate(). That's definitely not the case for String().

[ikasiuk]

() "Hello, " name "!" is, to my way of thinking, unacceptably ugly in a Ceylon-based template language.

You already mentioned that earlier and I didn't quite understand what you mean. Can you give an example of how that would look?

[gavinking]

Well, for example:

Div {
    Span { 
        Text { 
            () "Hello, " args.name "!" 
         }
    }
}

[ikasiuk]

Actually, what we need here is not a reference to a String() method but a reference to a String attribute, specifically to a getter. Is there a plan how attribute references will work in Ceylon?

[drochetti]

I have a suggestion here, but that would imply in changing the just released M5 string verbatim stuff. I know that it's been only two days, but I wasn't confortable with the previous syntax and still not entirely confortable with the new syntax (I can't even write markdown examples without escaping it, and I don't even know how to escape them). Maybe I'll get used to it in a while or maybe that's just because I'm brazilian and I hate the ` because of my mother language... but, well, let me give a try here:

The basic idea is that the { } play a key role in Ceylon. Much more than in C, Java, Javascript, etc. We use those for language basic and flow structures, named args constructor, new sequence shortcut syntax (anywhere else?). So I thought a natural way of interpolate strings, including lazy evaluation, would be:

value person = Person {
    ...
};

Html {
    Body {
        Div {
            "I'm { person.name } and I'm {
                => person.birthDate.timeUntil(today()).years() // too long expression, indent
            } years old.";
        }
        P {
            "what do you guys think? { => answer() }";
        }
    }
}

I like the readability of this code! =) Question is: is this syntax ambiguous to compiler?

So sorry to bring the whole string interpolation discussion back again...

ps: I know that the date/time code may be completely wrong, I haven't played with it yet, so this is just a dummy example.

Cheers!

[gavinking]

@drochetti

I can't even write markdown examples without escaping it

Use verbatim strings.

Question is: is this syntax ambiguous to compiler?

The issue is not that it's ambiguous as such. Only that it can't be very easily tokenized, since a } is ambiguous from the point of view of a stateless tokenizer.

Have you ever noticed how when you look at code examples involving string interpolation on the web, the interpolated expression is not correctly syntax-highlighted? But that on the Ceylon website, using our string interpolation syntax, it is? The reason is that online syntax highlighters use regexes, which can handle our syntax, but not stuff like #{ ... }.

It's also to my mind a problem that when you're typing an interpolated expression in the IDE, and you type an opening brace {, that the whole rest of the file would wind up incorrectly tokenized.

That's why I'm trying to avoid any syntax where the closing delimiter for an interpolated expression is also part of the expression syntax.

[gavinking]

Actually, I agree with @drochetti. I think we should try to come up with a syntax for delimiting lazily-interpolated expressions. That is, the difference between eager and lazy interpolation would not be the string delimiters, it would be the expression delimiters. But what syntax?

[gavinking]

This kinda works:

String() template = 
    "name: ``=>person.firstName`` ``=>person.lastName``
     age: ``=>person.age`` years";

Though I guess I could see someone arguing that the following is essentially the same and less verbose:

String() template = 
    =>"name: ``person.name`` ``person.lastName``
       age: ``person.age`` years";

[gavinking]

Let's deal with this in Ceylon 1.1, in conjunction with #491, if it turns out we really need them.