search

microsoft / vscode

Visual Studio Code
https://code.visualstudio.com
MIT License
160.95k stars 28.24k forks source link

shellscript tokenizes regex pattersn that contain (( incorrectly #139913

Closed lilyball closed 2 years ago

lilyball commented 2 years ago

Issue Type: Bug

Bash supports regexes with [[ foo =~ pattern ]]. The shellscript syntax handles these mostly correct, except when (( occurs in the pattern. It's legal for a pattern to contain (( but VSCode tokenizes it as a math expression instead.

Reproduction:

[[ foo =~ ^((ab|bc)+) ]]
Screenshot showing the tokenization. The two open parens are colored distinctly and the scope inspector shows that this begins string.other.math.shell

VS Code version: Code 1.63.2 (Universal) (899d46d82c4c95423fb7e10e68eba52050e30ba3, 2021-12-15T09:37:28.172Z) OS version: Darwin arm64 21.2.0 Restricted Mode: No

System Info |Item|Value| |---|---| |CPUs|Apple M1 Max (10 x 24)| |GPU Status|2d_canvas: enabled
gpu_compositing: enabled
metal: disabled_off
multiple_raster_threads: enabled_on
oop_rasterization: enabled
opengl: enabled_on
rasterization: enabled
skia_renderer: disabled_off_ok
video_decode: enabled
webgl: enabled
webgl2: enabled| |Load (avg)|2, 2, 2| |Memory (System)|64.00GB (23.75GB free)| |Process Argv|--crash-reporter-id 13a4da52-f255-450c-9b61-baf8cdc1b324| |Screen Reader|no| |VM|0%|
Extensions (56) Extension|Author (truncated)|Version ---|---|--- vscode-sievehighlight|adz|1.0.6 vscode-icalendar|af4|1.0.1 Bookmarks|ale|13.2.2 project-manager|ale|12.4.0 ascii-tree-generator|apr|1.2.4 troff|ban|1.0.22 vagrant|bbe|0.5.0 vscode-fish|bma|1.0.24 vscode-direnv|cab|1.0.0 vscode-modelines|chr|2.0.5 bracket-select|chu|2.0.2 doxdocgen|csc|1.3.2 viml|dun|0.1.7 gitlens|eam|11.7.0 EditorConfig|Edi|0.16.4 vscode-todo-plus|fab|4.18.4 lex-flex-yacc-bison|fau|0.0.3 macros|ged|1.2.1 vscode-pull-request-github|Git|0.34.2 vscode-graphql|Gra|0.3.41 fn-touchbar|Hap|1.0.0 beautify|Hoo|1.5.0 jenkinsfile-support|ivo|1.1.0 better-cpp-syntax|jef|1.15.10 nix-ide|jno|0.1.18 language-haskell|jus|3.4.0 swift|Kas|0.2.0 nim|kos|0.6.6 autoconf|mae|0.1.0 gluon|Mar|0.3.0 rust-analyzer|mat|0.2.867 vscode-elixir|mjm|1.1.0 remote-containers|ms-|0.209.6 remote-ssh|ms-|0.70.0 remote-ssh-edit|ms-|0.70.0 remote-wsl|ms-|0.63.13 cmake-tools|ms-|1.9.2 cpptools|ms-|1.7.1 cpptools-extension-pack|ms-|1.1.0 vscode-ios-common-files|Ort|1.0.5 synthwave-vscode|Rob|0.1.10 llvm|RRe|0.1.1 perl6-lsp|scr|0.3.0 just|ske|2.0.0 tcl|sle|0.2.0 ascii-plist|spe|1.0.3 rewrap|stk|1.15.4 even-better-toml|tam|0.14.2 shellcheck|tim|0.18.3 jest-snapshot-language-support|tle|1.1.1 tmlanguage|Tog|0.9.0 cmake|twx|0.0.17 vscode-terminal-here|Tyr|0.2.4 colonize|lil|0.0.1 FilterText|yhi|0.0.15 vscode-proto3|zxh|0.5.5 (1 theme extensions excluded)
A/B Experiments ``` vsliv368:30146709 vsreu685:30147344 python383cf:30185419 vspor879:30202332 vspor708:30202333 vspor363:30204092 pythontb:30283811 pythonptprofiler:30281270 vshan820:30294714 vstes263cf:30335440 pythondataviewer:30285071 vscod805:30301674 pythonvspyt200:30340761 binariesv615:30325510 bridge0708:30335490 bridge0723:30353136 vsaa593:30376534 pythonvs932:30410667 vscop804cf:30404767 vs360:30404995 vsrem710:30416614 vscexrecpromp3c:30407761 ```
alexr00 commented 2 years ago

We get our shellscript grammar from https://github.com/atom/language-shellscript; however, they are no-longer updating their textmate grammar. Linking to https://github.com/microsoft/vscode/issues/77675, which tracks investigating a new shellscript grammar.

lilyball commented 2 years ago

That issue has been open for three years. Can we not patch the current grammar to support (( properly? This is not an esoteric construct, and it breaks the rest of the file.