microsoft / vscode

Visual Studio Code
https://code.visualstudio.com
MIT License
164.25k stars 29.3k forks source link

Monarch throw error when nextEmbedded: '@pop' in action.cases field #172061

Open dmitriypereverza opened 1 year ago

dmitriypereverza commented 1 year ago

I created a local playground here https://microsoft.github.io/monaco-editor/monarch.html with the following:

return {
  defaultToken: "invalid",

  tokenizer: {
    root: [
      [/^(theme:)(.*)/, ["keyword", "identifier"]],
      [
        /^( +)(script:)/,
        ["text", { token: 'keyword', next: '@script.$1', 
        nextEmbedded: 'text/javascript' 
        }],
      ],
    ],
    script: [
      [/^( *)/, {
        cases: {
          '$1!~$S2\\s{4,}': 'text',
          '@default': { token: '@rematch', next: '@pop', nextEmbedded: '@pop', log: 'scriptEnd1 => "$1"' }
        }
      }],
    ],
  },
};
image

And in console Uncaught Error: monarch-language-mylang: no rule containing nextEmbedded: "@pop" in tokenizer embedded state: script.

But if i add rule [/```/, { token: '@rematch', next: '@pop', nextEmbedded: '@pop' }], in script error is gone.

image

Seems like monarch don't check nextEmbedded: '@pop' in action cases.

PR: #172193

ilyakarev commented 1 year ago

Confirm. Issue exists. Need that fix too.

timdmackey commented 1 month ago

I need this too. I'm working with an embedded language where the escape sequence to exit the embedded language happens inside the embedded language. Right now my highlighting can't work correctly because I need to highlight the escape sequence as part of the embedded language, not as part of the container language.