TMThemes are not working properly

[dev-drprasad]

I am not sure whether issue is related with monaco-textmate project or not. But, I don't know where to ask.

I am able to wire tmgrammers with monaco but colorization is not working as expected with tmthemes.

expected colors (screenshot taken from vscode): actual colors (screenshot taken from my project):

There is lot of difference between those two screenshots.

demo: https://codesandbox.io/s/monacowithtmgrammertmtheme-gc3o2

Any help is appreciated. Thanks

[zikaari]

You wrote perfect code!

The culprit is monaco-editor-textmate (wireTMGrammars), specifically this line of code.

When I wrote that code I wasn't very experienced with monaco.* API's so I did what I could at the time. Later I discovered someone's code and it blew me away!

The saviour is this write here:

editorInstance._themeService.getTheme().tokenTheme._match('punctuation.directive')

The current logic (or lack thereof) in monaco-editor-textmate will just return the second last token/scope from the array of all the scopes returned by grammar.tokenizeLine.

Now, what happens if there's no theme rule matching the second-last scope? Well you guessed it, it doesn't match and is rendered without any colors (or invalid colors).

To fix this we need to check if the scope we're returning does match against a rule in the current theme. And that's where the saviour code comes into play.

We start matching the returned tokens/scopes, starting from the last one and return the first one who's matchRes._foreground !== getTheme().tokenTheme._mainRule._foreground. Here _mainRule._foreground is the default style, i.e no match found.

See how this is done in codemirror-textmate

Remeber we're still returing the scope name, just the one that has a valid match.

I'd highly appreciate if you can create a PR in monaco-editor-textmate, update the API's as you see fit (you'll need editorInstance param in call to wireTmGrammars)

Let me know if you'd be willing to do this!

[dev-drprasad]

This is what i came up with

const TMToMonacoToken = (editor, scopes) => {
  let scopeName = "";
  // get the scope name. Example: cpp , java, haskell
  for (let i = scopes[0].length - 1; i >= 0; i -= 1) {
    const char = scopes[0][i];
    if (char === ".") {
      break;
    }
    scopeName = char + scopeName;
  }

  // iterate through all scopes from last to first
  for (let i = scopes.length - 1; i >= 0; i -= 1) {
    const scope = scopes[i];

    /**
     * Try all possible tokens from high specific token to low specific token
     *
     * Example:
     * 0 meta.function.definition.parameters.cpp
     * 1 meta.function.definition.parameters
     *
     * 2 meta.function.definition.cpp
     * 3 meta.function.definition
     *
     * 4 meta.function.cpp
     * 5 meta.function
     *
     * 6 meta.cpp
     * 7 meta
     */
    for (let i = scope.length - 1; i >= 0; i -= 1) {
      const char = scope[i];
      if (char === ".") {
        const token = scope.slice(0, i);
        if (
          editor._themeService.getTheme()._tokenTheme._match(token + "." + scopeName)._foreground >
          1
        ) {
          return token + "." + scopeName;
        }
        if (editor._themeService.getTheme()._tokenTheme._match(token)._foreground > 1) {
          return token;
        }
      }
    }
  }

  return "";
};

Improved colorization with above code:

It solves problem to some extent. But still, colors are not matching EXACTLY with vscode. You can compare with first screenshot. I dont get it why. We should check vscode implmentation, i think.