Fixed regex literals and tags, Added concrete syntax highlighting

[ahmadsalim]

You can see the effects of concrete syntax highlighting here: Concrete syntax highlighting

[PaulKlint]

Looks very good. I tried varies snippets and works well, thanks.

Below is an example that surprises me: (string literals like "template_open" are not highlighted in red, while they are in other contexts). Any clue?

private MuExp translateStringLiteral(s: (StringLiteral) `<PreStringChars pre> <Expression expression> <StringTail tail>`) {
    preIndent = computeIndent(pre);
    return muBlock( [ muCallPrim3("template_open", translatePreChars(pre), pre@\loc),
                      *translateExpInStringLiteral(preIndent, expression),
                      *translateTail(preIndent, tail),
                      muCallPrim3("template_close", [], tail@\loc)
                    ]   );
}

[ahmadsalim]

@PaulKlint Thanks. Hmm, I might have been to general in scope for the lexical [] so it also apparently included list literals. I will try and see how to fix this.

[PaulKlint]

Here is another one: int n versus list[int] n. In the former int is green in the latter it is not. Is there a reason for this?

[ahmadsalim]

Yeah, so the major issue is that [] seems to have a different meaning when it's inside lexical or syntax than when it's outside. I have to figure out a nice way to capture this property.

This is because e.g. [/bla/] is a list containing a regex in a normal context, whereas it is something which expects any of the characters / b l or a in a lexical context). The current way is that it treats all [] uniformly as lexical contexts (if it doesn't it will ruin the highlighting elsewhere in the grammar files).

[ahmadsalim]

It's my first time writing a TextMate grammar, so it's taking me some time to figure out how to do these things correctly.

[PaulKlint]

This has been on our list for a long time since we are no experts on this either. I can imagine that some more regexp fiddling is needed here and that it is not so simple to figure out what these cooperation regents do precisely.

Since I find your highlighter so good (on par or even better than our Eclipse one a I would say) I am doing more experiments and will report surprises here (take this as a compliment, its intended to make this really good!).

Here is another one where a comment is missed:

// comment 1

n = 1;  // comment 3
l = [1, // comment 3: non highlighted
     2, 
     3];

[ahmadsalim]

Great, thank you for the examples and the comments! I will take a look at fixing this tomorrow, I think I have an idea how to proceed.