Bracket Pair Colorization does not handle Perl punctuation variables

[FractalBoy]

Type: Bug

Perl has a bunch of punctuation variables including $) and $(. You can find documentation on all of these variables in perlvar.

Bracket pair colorization does not seem to handle these well, resulting in it turning brackets red when there is actually a matching close bracket.

To reproduce:

1. Turn on bracket pair colorization
2. Create new perl file (test.pl, for example)
3. Write the following code:

   #!/usr/bin/perl
   ($));

4. Run the code in perl - it runs with no syntax errors (though does nothing).
5. VSCode reports the parentheses as not matching.

Here is another, more complex, example - in this case, VSCode colors all the curly brackets red instead of the parentheses:

#!/usr/bin/perl

sub test
{
    return;
}

test(
    sub {
        my $x = sub { scalar(getpwuid($>)), $) };
    }
);

VS Code version: Code 1.73.1 (6261075646f055b99068d3688932416f2346dd3b, 2022-11-09T04:27:29.066Z) OS version: Windows_NT x64 10.0.19043 Modes: Sandboxed: No Remote OS version: Linux x64 4.18.0-348.12.2.el8_5.x86_64 Connection to 'ssh-remote+vm-mreisner' could not be established

System Info

|Item|Value| |---|---| |CPUs|Intel(R) Core(TM) i5-8350U CPU @ 1.70GHz (8 x 1896)| |GPU Status|2d_canvas: enabled
canvas_oop_rasterization: disabled_off
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
skia_renderer: enabled_on
video_decode: enabled
video_encode: enabled
vulkan: disabled_off
webgl: enabled
webgl2: enabled
webgpu: disabled_off| |Load (avg)|undefined| |Memory (System)|15.88GB (0.75GB free)| |Process Argv|--crash-reporter-id 11265485-e23d-4f5a-b75b-ed9aa0dd293e| |Screen Reader|no| |VM|33%| |Item|Value| |---|---| |Remote|SSH: vm-mreisner| |OS|Linux x64 4.18.0-348.12.2.el8_5.x86_64| |CPUs|Intel(R) Xeon(R) Platinum 8268 CPU @ 2.90GHz (2 x 2893)| |Memory (System)|1.93GB (0.64GB free)| |VM|100%| Connection to 'ssh-remote+vm-mreisner' could not be established

Extensions (36)

Extension|Author (truncated)|Version ---|---|--- RPMSpec|1do|0.1.0 better-cpp-syntax|jef|1.16.3 jupyter-keymap|ms-|1.0.0 remote-containers|ms-|0.262.3 remote-ssh|ms-|0.92.0 remote-ssh-edit|ms-|0.84.0 remote-wsl|ms-|0.72.0 vscode-remote-extensionpack|ms-|0.23.0 remote-explorer|ms-|0.0.2 vim|vsc|1.24.3 blamer-vs|bea|0.5.2 doxdocgen|csc|1.4.0 xml|Dot|2.5.1 pls|Fra|0.0.16 todo-tree|Gru|0.0.220 haskell|has|2.2.2 plantuml|jeb|2.17.4 better-cpp-syntax|jef|1.16.3 svn-scm|joh|2.15.5 cmake-language-support-vscode|jos|0.0.7 language-julia|jul|1.38.2 language-haskell|jus|3.6.0 vscode-dotnet-runtime|ms-|1.6.0 isort|ms-|2022.8.0 python|ms-|2022.18.2 vscode-pylance|ms-|2022.11.40 jupyter|ms-|2022.9.1303220346 jupyter-keymap|ms-|1.0.0 jupyter-renderers|ms-|1.0.12 vscode-jupyter-cell-tags|ms-|0.1.6 vscode-jupyter-slideshow|ms-|0.1.5 cmake-tools|ms-|1.12.27 cpptools|ms-|1.12.4 cpptools-extension-pack|ms-|1.3.0 hexeditor|ms-|1.9.9 cmake|twx|0.0.17 (2 theme extensions excluded)

A/B Experiments

``` vsliv368:30146709 vsreu685:30147344 python383:30185418 vspor879:30202332 vspor708:30202333 vspor363:30204092 vswsl492:30256859 vstes627:30244334 vslsvsres303:30308271 pythonvspyl392:30443607 vserr242cf:30382550 pythontb:30283811 vsjup518:30340749 pythonptprofiler:30281270 vshan820:30294714 vstes263:30335439 vscorecescf:30445987 pythondataviewer:30285071 vscod805:30301674 binariesv615:30325510 bridge0708:30335490 bridge0723:30353136 cmake_vspar411:30581797 vsaa593:30376534 pythonvs932:30410667 cppdebug:30492333 vsclangdf:30486550 c4g48928:30535728 dsvsc012:30540252 azure-dev_surveyone:30548225 vsccc:30610678 pyindex848:30577860 nodejswelcome1:30587005 282f8724:30602487 gswce1:30612156 3d0df643:30613357 dbltrim-noruby:30604474 89544117:30613380 cc2-nonfim-prod:30622702 ```

[hediet]

This is due to perl having the variable $). Perls grammar should be tweaked so that this token is skipped by bracket pair matching.

[AbdulMateenzwl]

This all is due to $ symbol as Currently Vs Code does not recognize it .There is need to do minor changes to resolve this issue.

[debadree25]

Hello @hediet is it possible to know which part of the codebase to look into fix this? will try to do it then

[antonioprudenzano]

The grammars are pulled from textmate/perl.tmbundle repository, shouldn't this be handled by them?

[ordabool]

Hi, will try to work on it in the coming days. Will update :)

[ordabool]

Hi @hediet, From what I can tell, it looks like this can be solved by using a negative lookbehind (for $) in the regex that looks for brackets.

This is handled in src/vs/editor/common/model/bracketPairsTextModelPart/bracketPairsTree/brackets.ts, and I managed to fix the behavior using the following code (and also passing languageId to BracketTokens from the static createFromLanguage function):

    getRegExpStr(): string | null {
        if (this.isEmpty) {
            return null;
        } else {
            const keys = [...this.map.keys()];
            keys.sort();
            keys.reverse();
            const regexStr = keys.map(k => prepareBracketForRegExp(k)).join('|');
            if (this.languageId === 'perl') {
                return '(?<!&)' + '(' + regexStr + ')';
            }
            return regexStr;
        }
    }

However, I'm not comfortable with this solution being hard coded, so maybe a better approach is to add new field to perl.language-configuration.json that will handle negative lookbehind. This will cause a bigger change to the codebase, so I prefer to get your opinion first.

(Note: I know that the current solution also won't interpret } in {$} as a bracket. Once I'll know if I should create a general or specific solution, I will fix it.)

Thanks!

[hediet]

This could also potentially be solved with adjusting the textmate grammar, so that $) gets a special scope that is marked as not containing brackets (which is also what I suggested earlier).

I wouldn't allow brackets to have a negative lookbehind, especially if this can be fixed in the grammar.

[ordabool]

Hi @hediet , thanks for your response.

I tried defining a new scope in perl.tmLanguage and use it in the "unbalancedBracketScopes" property as seen here https://github.com/microsoft/vscode/issues/144736 , but it didn't work.

I don't really understand how to debug this behavior. Sometimes when I launch my debug VSCode no code is colored, sometimes just parts of it are colored (even though no changes to the code has been made between debugs).

Can you please guide me on how to proceed? Thanks!

[anshuman0123]

please assign me this issue

[ashutosh887]

I'm willing to work on this

[meganrogge]

@hediet I think this should be on the May milestone?