[Language request] Rust

[Geobert]

Thank you for this plugin.

However I'm only using Rust on VSCode :)

Can we please have support for Rust?

[EvgeniyPeshkov]

Hello, @Geobert . I would be glad to add support for Rust. Unfortunately I do not familiar Rust, but, assuming it is syntactically similar to C++, I think I can manage this. In the end of a day all programming languages are the same (=. Could you please send me any complex source. The more complex - the better.

[Geobert]

Hi, thanks for considering it!

From the top of my head: https://github.com/serde-rs/serde https://github.com/seanmonstar/warp https://github.com/actix/actix-web https://github.com/cobalt-org/cobalt.rs https://github.com/cobalt-org/liquid-rust

this should provide you a good reference :D

[EvgeniyPeshkov]

Hi @Geobert , I suppose I'll add Rust in a day or two. Please, be patient. (=

[EvgeniyPeshkov]

Hello, @Geobert . It's done. Please, check out the new version 0.2.2. It includes Rust support. In VSCode execute the next command: Ctrl+P -> ext install evgeniypeshkov.syntax-highlighter. Please, don't hesitate to tell me if anything is highlighted wrong. I've tried my best, but still, I'm not Rust user.

[Geobert]

Hi @EvgeniyPeshkov ,

Thank you for this!

Trying to customize it and it doesn't seems to take my setting into account:

   "workbench.colorTheme": "Dark+ One Monokai",
    "editor.tokenColorCustomizations": {
        "[Dark+ One Monokai]": {
            "types": "#56b6c2",
            "functions": "#98c379",
            "variables": "#e06c75",
            "strings": "#e5c07b",
            "comments": "#777d88",
            "keywords": "#e06c75",
        }
    },

[EvgeniyPeshkov]

Hi @Geobert , You should set "syntax.types": "#56b6c2", etc., to redefine extension's colors. Details and example can be found here.

[Geobert]

Oh, I used the second example (without texmate section as it was not working, I tried a minimal version)

[EvgeniyPeshkov]

Just in case, the second example explains how to synchronize TextMate and Syntax Highlighter colors, and it's only for C++. Unfortunately, there are tons of TextMate tokens, and they are different for various languages.

[Geobert]

I see :D I'm configuring the extension and I'll let you know if anything needs changes :)

[Geobert]

First wave of obvious changes :) With these colours to make everything different:

{
            "syntax.type": "#5744ff",
            "syntax.function": "#eaff73",
            "syntax.variable": "#c5c8ce",
            "syntax.string": "#e5c07b",
            "syntax.comment": "#777d88",
            "syntax.keyword_directive": "#00a808",
            "syntax.keyword_constant": "#e756fa",
            "syntax.keyword_control": "#e06c75",
            "syntax.keyword_operator": "#ffa939",
            "syntax.storage_modifier": "#fc0015",
            "syntax.punctuation": "#007c72",
            "syntax.number": "#c678dd",
            "syntax.namespace": "#1f6788",
        },

use directive

in this screen, in use directive, names that are not a keyword like self or crate or super can be considered as namespace. So db is wrongly highlighted as a type, FakeConnection is wrongly coloured as a variable. In a use directive, as defined a new name for a type, so maybe it should be coloured as a type as well (reffering to Connection in as Connection)

We can put functions in a use directive as well, here transaction is a function and coloured as a type.

In Rust, thing in #[] are kind of directives as well. Here cfg, not are directive-like and test is a value.

namespace and enum variant

here db::session is namespace-like and coloured as a type

In Rust, enums have variants and the type Result has 2 variants, Ok and Err. Here Ok is considered like a function, I think it should be something else. Type? Not sure, but I'm sure it shouldn't be function (confusing).

Unit value (void like)

In Rust, we can note the void value, called Unit, as (). I don't know if it's possible though, to differentiate () as a value (coloured as a number maybe? as true and false uses number colour) and () as no parameter in a function call.

[EvgeniyPeshkov]

Could you kindly send me this sources, so I can make screenshots and send them back. Most things can be fixed, but I'm not sure about all of them. Thank you in advance.

[Geobert]

These screen are from: https://github.com/ghost-in-the-sushi/efficio-server/blob/master/src/db/aisles.rs

Thank you again for your work!

[EvgeniyPeshkov]

use and namespaces and directive

I believe those are better:

And even Unit is possible

I'm still not sure about enums like Result and functions within use-directive.

[Geobert]

Way better! I'm not sure about test in #[] as this is a value (vs cfg being a "directive")

[Geobert]

Oh and in use, redis is still coloured like a variable, instead of namespace

[Geobert]

Another idea: Atom is supporting Rust with Tree-sitter as well, maybe you can compare the results with Atom as well?

[EvgeniyPeshkov]

Yes, I've used their grammar definition as basis (=. In fact I think we've already done this a little bit better. For example, in Atom there is no namespace, everything is type. As you might noticed, I've also made no difference between namespace and type at first time (=.

[EvgeniyPeshkov]

And speaking of code like Ok(false). It's very interesting question. I would say philosophical. How we should treat constructors? They are in fact functions that just have the same name as type and return variable of this type (=. So in some sense it's natural to highlight them as functions, constructor is not a type. Moreover, in almost all languages in most cases there is no syntactical difference between function call and constructor. The same happens in C++.

[Geobert]

On constructor, I agree for the most part as I didn't mentioned AisleId(...) call for instance. But for this Ok(true) case, I'm more mitigate. My personal taste want to distinguish this in particular because in Rust, error management is done by this enum Result and is very idiomatic. Same goes for the enum Option which has 2 variants: None and Some(value).

I tried to gather more Rustcean here by promoting your extension in /r/rust. I'm not a Rust specialist, just an enthusiast, and there is large part of the language I didn't use (macro definition, for example).

[repax]

It makes most sense to highlight all enum variants like types:

enum Message {
    Quit,
    Move { x: i32, y: i32 },
    Write(String),
    ChangeColor(i32, i32, i32),
}

They are not like constructors (functions) in the C++ sense because there's no code within. Normal functions are used when internal logic is required for construction: fn new_foo(x: i32) -> Foo. You also use the variants in patterns like:

if let Ok(v) = result {...}

which are also not function calls.

[EvgeniyPeshkov]

Hello, @Geobert . First of all, thank you for your active contribution.

I'll try to be brief answering your suggestions. They are all somewhat related. Things like Result{OK, Error}, while maybe very conventional, are still not part of core language. Tree-sitter, for example, doesn't treat them specially. So the only way to highlight them differently, is to expand every term, and see what is inside, either it's OK or SomeType. This will significantly impact performance. Just imagine how many terms could be in source file. And for almost all of them we will try to find match in list of special words. Nearly the same situation is with exporting functions alongside types in {use list}. Tree-sitter by itself is only incremental syntax parser. It doesn't know anything about kind of term, either it type, or enum, or function, this is task for language servers. It knows things like "in call expression we expect this term to be function". To deal with this, Atom additionally uses RegEx on top Tree-Sitter. When it parses something like this use redis::{self, transaction, Commands}, it simply assumes that everything starting with capital letter is type. Looking at its grammar definition, it seems to me, Atom won't highlight functions in {use list} at all. So, basically, Atom just relies on code conventions, and if anyone declares type starting from lowercase letter, everything will break apart. For now I cannot afford adding regex parser for the same performance reason.

So here is my proposal, and I think it's reasonable. In exports we will highlight all outer namespaces as namespaces, and what we actually export as types. Maybe it will not reflect underlying kind of term, bit this is very convenient because we emphasize relevant part. Personally, I prefer to assign to namespaces the same color as types, but make it a little bit dimmed. This will look like this:

Looks good to me. In fact it's just matter of habit. An hour with such highlighting and you'll get used to it. The same will apply to other code

[EvgeniyPeshkov]

Hello, @repax . I think you got me wrong. I was just saying that constructors are essentially functions. Of course I do not insist that members of enum are not types (=. I've fixed this:

[EvgeniyPeshkov]

So for now, I suggest to publish updated version with, at least, a way better support of Rust. Then we will see how it can be improved. Sounds right?

[Geobert]

For the functions in use, it's not a big deal anyway.

Be aware that as Rust provide tools to keep formatting and naming convention clean, there is a kind of dominating coding convention in the community. It can be tweaked of course, but as far as I know, few are doing it.

For the enum's variants, it is correctly coloured in their declaration, the doubt is when we use them in the code as a pattern or result of a function.

Nonetheless, I agree to publish now, let the community play with it and see what can be improved :)

[EvgeniyPeshkov]

I've published the new version, you can check it out. I'm aware that there are tools promoting naming convention, and I believe well-written code follows it. At the moment, I'm just not ready to extend parsing engine. It will require some work. Moreover I need to be sure that performance will not be impacted, at least for other languages. Can you elaborate on use of enums "in the code as a pattern or result of a function".

[optimalstrategy]

@EvgeniyPeshkov By "in the code as a pattern or result of a function" @Geobert probably meant usage of enums in pattern matching. Here is a chapter on patterns from the book. There is also a reference that describes pattern syntax.

The highlighting looks well, but I would suggest to highlight Results's Ok and Err, and Option's Some and None (which are enum variants) in a different color, because they're considered idiomatic, and used by most, if not all, Rust programs. It may also be useful to highlight frequently used traits: Into, From, ToString, Copy, Clone, Default, AsRef, AsMut, Iterator, IntoIterator, Send, Sync, as well as primitive types bool, char, i8, i16, i32, i64, i128, isize, u8, u16, u32, u64, u128, usize, f32, f64, and str.

Also, it would be great if there were an option to highlight lifetimes as a single item. For example:

impl struct S<'ctx, Type> { // I would like the ' and `ctx` to be highlighted in the same color 
    fn method(ctx: &'ctx Type) { ... }
}

[Geobert]

Can you elaborate on use of enums "in the code as a pattern or result of a function".

It's as @OptimalStrategy said :)

For the lifetime, I don't know if tree-sitter distinguish it, so it might not be possible.

I'm not available tonight to test the new version, but I'll will do tomorrow!

Let's close this issue for now, and I'll create more targeted issue if needed

[EvgeniyPeshkov]

I agree with @Geobert on closing this issue. I consider that initial support of Rust is implemented and discussion of minor fixes can be continued in more targeted topics for convenience. @Geobert , will meet you in 5 issues, you've already opened (=. Big thanks to everyone for participation.