Grammar Issue with additive operations on string

[TheCrether]

Hello,

I have been doing some work with this project and found a weird grammer error (for linting). When a plus sign (+) is right in front of a double quoted string, the parser/grammar does not recognise it as a additive operation and afterwards marks everything as a string until a certain point. I have tried looking at the MagikGrammar.java on where the incorrect behaviour exactly occurs, but I am not sure. My guess it is around the UNARY_EXPRESSION rule (link to the code)

I have attached a code snippet to test it on and a screenshot of what it looks like for me. This is quite old code and not formatted but I would like "correct" highlighting on it for searching things of course 😅 and because I am not allowed to change it.

_pragma(classify_level=restricted,
        topic={ubisense, e_strom_trasse, record_exemplars})
_method e_typ_parameter_editor_plugin.init_actions()
    ## ...removed code
    _loop
        _if _not _self.actions.includes?( action_name )
        _then
            _self.add_action(
                sw_action.new(action_name,
                          :caption, _self.message(action_name)+"...",
                          :short_help_text, _self.message(action_name+:_sh),
                    #      :image, { action_name, _self.image_module_name },
                          :enabled?, _false,
                          :engine, data[1],                      
                          :action_message, data[2]))
        _endif      
    _endloop
_endmethod
$

##
### 500+ LINES OF CODE IN BETWEEN
##

_pragma(classify_level=advanced,
        topic={isoware, e_strom_trasse, e_cross_section, e_editors})
_method e_typ_parameter_editor_plugin.parameter_page_selected(num)
    ## Page with number NUM is selected. make the button "copy
    ## parameters" visible, if num>2 , and invisible otherwise.
    ##
    .item_hash[:copy_values].visibility << num>2 _andif
                           .val_rope _isnt _unset _andif
                           _not .val_rope.empty? _andif
                           .item_hash[:pseudo_field_listers][num] _isnt _unset _andif
                           _not .item_hash[:pseudo_field_listers][num].empty?

_endmethod
$

[StevenLooman]

Thank you for this issue @TheCrether. Good to see yet another user of magik-tools! 👍

Which version (of the language server) are you using? I cannot seem to reproduce your results. This is what I'm seeing, with the current develop branch (which should be the same as the latest release regarding the Magik grammar):

In your screenshot, I do see that some keywords are red instead of the default VSCode-theme (I think) purple. Did you configure this yourself? Perhaps there are syntax errors (or rather, something the grammar cannot parse) before this code block.

[TheCrether]

Interesting, I am using a changed version of magik-tools, which I plan to upstream to this branch since I have made some changes that I think are good as a whole. The extension I use is also custom and I'll be looking into either merging it into magik-tools or creating a new repository since it also offers features without magik-tools. The red accent is from trying out the bracket colorization feature with the keywords like _loop and _endloop

I forgot that I have not merged the latest commits from you since I didn't touch this project in a few months 😅.

I now built the extension from your current develop and now I do not have the problem with the wrong parsing but the string in question still does not get treated like one:

Any ideas on why?

[StevenLooman]

Judging from the screenshot, more tokens are not getting properly marked, such as comments, and some of the keywords. I would check the language server output (either from VSCode itself in the OUTPUT panel --> Magik Language Server, or magik-language-server.log in your home directory) for any errors. If there was an error during the semantic tokenization (or how would one call it), it will most likely fall back on the TextMate grammar.

[TheCrether]

Ok, so I realised another error on my side because apparently the version could not find my JAVA_HOME variable so I had to set the magik.javaHome setting manually. Now the semantic tokenisation is correct :)

Still, the TextMate grammar should also "work correctly" if the language server is not running. I will update this if I find a better TextMate definition that can handle this case.

[StevenLooman]

Great you were able to find the cause!

To be honest, I'd rather get rid of the TextMate grammar. It caused confusion in your case. Also, it requires maintenance time (or rather, should require maintenance, which it does not get). Time I'd rather invest in other things.

[TheCrether]

Fair point, only RegEx for such things is not very useful in many cases 😅. I will be closing this issue until I find a better TextMate configuration. Thanks for your help!