Infer custom keywords based on definition files

[tamasfe]

We now have custom operator keywords from rhai definition files in the HIR, we can use this information to (re)parse scripts with these keywords known and then provide hover/semantic tokens or even completion for them in the LSP.

• [x] Use operators from definition files.
• [x] Semantic tokens.
• [x] Docs on hover.
• [ ] Go to definition? I left it out on purpose, I don't think the user typically wants to know where an operator was defined and it would be probably confusing as well.
• [x] Completions, I'm planning a completions-oriented sprint where this will be addressed among other things.
• [x] Custom precedence. We need to decide a syntax on this in definition files, right now all custom operators just use the lowest possible binding powers.

[tamasfe]

I made some progress on this as reflected on the OP. This is generally usable with some unresolved questions.

We could easily use precedence set in definition files, but I'm wondering how to represent them syntactically.

I see a few ways to do it, and I don't have a strong opinion on either:

Use Go-style parameters:

op (1, 2) is(?, ?) -> bool;

Or use generic param style:

op is<1, 2>(?, ?) -> bool;

Or introduce attributes:

#[precedence(1, 2)]
op is(?, ?) -> bool;

Or the one I like the least is to put it in docs and somehow parse it from there.

I don't like this because docs are not always properly parsed, and precedence is an integral part of operators anyway.

/// precedence: 1,2
op is(?, ?) -> bool;

[tamasfe]

I decided to go with the following syntax for this for now:

op is(?, ?) -> bool with (1, 2);

The respective optional binding powers are 1 and 2 after the with keyword.

The fact that it's after the definition and separated with the with keyword makes it easier to parse and unambiguous.

[schungx]

However, I only use one precedence value for each operator... do we need to split into two binding powers?

[tamasfe]

However, I only use one precedence value for each operator... do we need to split into two binding powers?

No, if associativity is not configurable, I can also remove the second one, it's optional right now anyway.

These are also not needed for built-in operators, the precedences for those are hard-coded.

[tamasfe]

I'll leave the second binding power in, users (and generated code) will just use the first one, but with it we have the ability to still define right-associated operators if needed in the future.

Also all information is available via hover and completion, I'd rather not implement go to definition for this as it would make the operators act too much like variables. We can add a separate command like go to operator for this if needed.

With this being said, I think this is done.