dart-lang / dart-syntax-highlight

Tools and documentation for how Dart code is formatted
BSD 3-Clause "New" or "Revised" License
22 stars 11 forks source link

Analyzer barfs parsing .. spreads? I'm not sure, included a minimal repro. #63

Open matanlurey opened 5 months ago

matanlurey commented 5 months ago
% dart --version
Dart SDK version: 3.3.1 (stable) (Wed Mar 6 13:09:19 2024 +0000) on "macos_arm64"

The following code does successfully execute, and gets highlighting, but depending on the tool (VSCode, Dartpad, etc) the syntax highlighting is off.

void main() {
  final code = StringBuffer();
  final data = null as dynamic;
  code.writeln(
    'final cards = <CardOrVariant>${_createList([
          ...data.cards.map(
            (c) => _invokeConstructor(
              'CanonicalCard',
              {
                'card': _nameToIdentifier(c.title),
              },
              indent: '  ',
            ),
          ),
          ..._collectVariants(data.cards, count: data.count),
        ])};',
  );
}

String _createList(
  Iterable<String> elements,
) {
  throw UnimplementedError();
}

String _nameToIdentifier(String name) {
  throw UnimplementedError();
}

String _invokeConstructor(
  String name,
  Map<String, String> fields, {
  String indent = '',   
}) {
  throw UnimplementedError();
}

Iterable<String> _collectVariants(
  Iterable<Never> cards, {
  required int count,
}) {
  throw UnimplementedError();
}

Example in VSCode, the trailing ])};', all look like they are in a string, but they are within ${...} blocks.

image

Dartpad looks worse (even more of the expression is highlighted as a string):

Screenshot 2024-03-20 at 5 11 12 PM

lrhn commented 5 months ago

It's not the cascade, its the embedded }. You can see it from:

void main() {
  var s = "${foo([
      "a",
      {},
  ])}";
}

The same issue happens here, and it goes away if you change {}, to // {},.

It probably only confuses the highlighter, which in VSCode is (I think) uses a separate grammar, and is not using the analyzer.

DanTup commented 5 months ago

It probably only confuses the highlighter, which in VSCode is (I think) uses a separate grammar, and is not using the analyzer.

VS Code actually uses both! It uses the textmate grammar (from this repo) initially - VS Code itself parses that and applies it. This gives very fast (but not entirely accurate) highlighting. And then it asynchronously asks the server for "semantic tokens" that get laid over the top (and this is why you may see the colours change shortly after opening a file).

Ideally these implementations would be very close, but in practice the textmate grammar is way simpler and often leaves things uncoloured for the semantic tokens to pick up. Interpolated expressions is one place that it doesn't work too well, although your example looks particularly bad. My guess is that it's because it doesn't handle the } (that is not the end of the interpolated expression) well.

I'll see if I can improve things.

DanTup commented 5 months ago

Actually, I'm not really sure I understand what the issue is. Here's what I see for both semantic tokens (left) and textmate grammar (right).

image

There's definitely something a little weird about textmate grammar because some parts of the expressions are coloured but mostly it's just treated as a string (probably we should treat the whole thing as a string, because I don't know if we can parse the code in the expression reliably in a textmate grammar). The closing characters are coloured like the opening equivalents though.

The semantic tokens (left) looks similar to @matanlurey's... The ])} are also coloured the same as their matching opening characters. There's some different in our themes (I'm using VS Code's Dark+ which seems to have more colours showing), but the string literal parts are red/brown and the dollar/braces/brackets are consistently white.

@matanlurey can you confirm what you're expected to see (and also what theme you're using in VS Code? It's not clear to me if your screenshot was directly from VS Code as it looked prettier than just a screenshot of the editor!).