Syntax highlighting gets confused in a few corner cases

[yminsky]

A few examples of the syntax highlighting not working quite right.

In the following example, you can see that begin/end are not highlighted as keywords --- they should ideally be in the same color as match and with.

Two issues are on display here: first, the first let is unhighlighted, presumably due to the [%map_open annotation. The second is that the "and" looks different from the "let", and presumably should be in the same syntactic class, since it's a keyword in the same construct.

[yminsky]

Similarly, struct and end are not marked as keywords, but probably should be.

[ghost]

Thanks for the report.

Two of these are definitely bugs that I can fix:

• let not being highlighted in [%map_open let
• struct/end not being highlighted after include

The others are "intentional" although I agree they look less than ideal with that theme.

The reason that some of the keywords begin, end, and and are highlighted differently from the others is because I've tried to optimize colorization for readability (via contrast) rather than the strict syntactic category of the token. I realize this is a somewhat controversial choice but there is at least a rationale behind it.

Here is what those snippets look like with the Dark+ theme that ships with VS Code:

The reason there's sometimes a lack of consistency between the themes has to do with the rather unfortunate and haphazard manner in which TextMate themes have evolved. There are really no standard scopes for highlighting different parts of the syntax.

There are some very rough conventions used for basic things like literals, strings, etc., but it's hard to rely on those. Some themes support much more fine-grained colorization than others. Some themes may provide scopes for certain syntactic constructs (sometimes even specific to a particular language) while others may not.

If you try out several different TextMate themes (especially across the different editors that support them) you'll probably notice a lot of little inconsistencies like this. I do try to account for the inconsistencies by tuning the highlighter with a number of popular themes and that works okay but the results are not perfect.

However, I will try to adjust the highlighting with Dracula for begin and end and see if there are some other areas I can improve.

For what it's worth, I expect that the Dark+ theme will usually look the best out of the available options because it tends to be more complete and has a wider color palette.

[yminsky]

I tried switching to Dark+, and I don't think it really helps. A few thoughts:

• I think and should really match let and in -- having them be similar helps you match up what is going on in a parallel binding. The current highlighting makes parallel bindings look kind of haphazard in my mind. I've never seen an OCaml highlighting scheme make this choice.
• Making begin/end look the same as match is more controversial. My main argument is that it makes the begin match ... with ... end pattern look more like a single structure, which seems good to me, since it's good practice to bound inner matches with begin/end.

As a general matter, I think font coloring schemes often lean too far in the candy-striper direction, with more colors than the reader's mind can easily grasp. I think leaning a bit minimalistic will lead to a more usable result.

[ghost]

I have fixed the [%map_open let issue in release 1.0.21. I am considering the other changes but they require some subtle refactoring of the grammar and I haven't yet had a chance to go through and make those changes and ensure they don't lead to more bugs.