search

microsoft / vscode-textmate

A library that helps tokenize text using Text Mate grammars.
MIT License
581 stars 115 forks source link

ModernPerl syntax error #64

Open mrGibi opened 6 years ago

mrGibi commented 6 years ago

Hi, I tried importing to VS Code a syntax from Sublime's ModernPerl. Syntax is in .plist file, attached here: Perl.commented-bad.txt (as .txt becouse of GitHub...). There are 3 problematic scopes in this file. They are commented out and VS Code works and colorize the Perl code, but if I uncomment them, then I get:

ERR empty range in char class: Error: empty range in char class
    at createOnigScanner (/Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar/vscode-textmate/release/main.js:1346:12)
    at RegExpSourceList.compile (/Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar/vscode-textmate/release/main.js:1399:30)
    at IncludeOnlyRule.compile (/Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar/vscode-textmate/release/main.js:1492:45)
    at matchRule (/Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar/vscode-textmate/release/main.js:2240:28)
    at matchRuleOrInjections (/Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar/vscode-textmate/release/main.js:2256:23)
    at scanNext (/Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar/vscode-textmate/release/main.js:2346:17)
    at _tokenizeString (/Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar/vscode-textmate/release/main.js:2339:9)
    at Grammar._tokenize (/Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar/vscode-textmate/release/main.js:2107:25)
    at Grammar.tokenizeLine2 (/Applications/Visual Studio Code.app/Contents/Resources/app/node_modules.asar/vscode-textmate/release/main.js:2077:22)
    at e.tokenize2 (file:///Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/workbench.main.js:3254:428)

Same file worked in Sublime perfectly. Any ideas what is causing the problem?

alexdima commented 6 years ago

It might be a difference in oniguruma versions. That error appears to be coming from oniguruma (regex engine powering all TM grammars interpreters). I believe that SublimeText is perhaps using a different version of oniguruma or perhaps they have forked it ...

msftrncs commented 5 years ago

If there is no difference in scoping (and that could be hard to find) then it might be as simple as Sublime catches the exception and ignores it. I noticed some class constructs where they are using captures in the class, what if the captures didn't match? There are a lot of weird constructs in this file. Isn't [\d\D] the same as .? Isn't [^}\}] the same as [^}]?

It would help if the oniguruma exceptions would be caught and some context logged as to what patterns were being attempted, and possibly not interrupt the remaining tokenizing.