[Rust] Please add Rust language support

[Arcitec]

Rust is a quickly emerging new language, which is like a very productive, memory-safe version of C. It's fantastic. Sadly monokai's support for it is incomplete, since the Rust language defines many custom text tokens.

Related ticket: #154

Here's a partial example file (not valid code, just meant as a test of syntax and it doesn't include all syntax offered by rust) where you can see that the syntax highlighting has many problems in Monokai Pro.

Examples of problems:

• struct and impl keywords are some weird light blue color, instead of being reddish/pinkish like fn, const, let, mut and other language keywords in Rust (and in other languages that Monokai fully supports).
• macros (anything that ends in !) such as the println!() are the same color as regular functions... this may or may not be a problem, but look at the Dracula Official theme, which colors them differently. I am not decided on which approach I prefer. Worth thinking about...
• To be honest, it's mostly perfect already. Just a few rust-specific keywords/concepts that need extra tweaking. The file I am providing below is NOT comprehensive and does not cover the whole language, so you'll need to research more files to see the full syntax offered. (edit: The example has been updated to cover every Rust keyword and should be close to 100% language coverage now)

use futures::executor::block_on;

#![feature(try_blocks)] // "try": not part of the language officially yet

use std::num::ParseIntError;

let result: Result<i32, ParseIntError> = try { // "try" keyword
    "1".parse::<i32>()?
        + "2".parse::<i32>()?
        + "3".parse::<i32>()?
};
assert_eq!(result, Ok(6));

#[cfg(test)]
mod tests {
    #[test]
    fn it_works() {
        assert_eq!(2 + 2, 4);
    }
}

mod network {
    fn connect() {
    }
}

extern crate my_crate as thing;
pub(crate) use std::io::Error as IoError; // "use" keyword (and "pub" and "as")
pub(crate) enum CoolMarkerType { }
pub struct PublicThing {
    pub(crate) semi_secret_thing: bool,
}

async fn hello_world() { // "async" keyword
    println!("hello, world!");
}

fn foobar() {
    let future = hello_world(); // Nothing is printed
    block_on(future); // `future` is run and "hello, world!" is printed

    let song = hello_world().await; // "await" keyword
    sing_song(song).await;
}

static N: i32 = 5;

static NAME: &'static str = "Steve"; // static keyword AND 'static lifetime

impl Add<Foo> for Bar {
    type Output = BarFoo;
    // omitted
}

type T = HashMap<i32,String>; // "type" keyword

trait PrintInOption {
    fn print_in_option(self);
}

// Because we would otherwise have to express this as `T: Debug` or 
// use another method of indirect approach, this requires a `where` clause:
impl<T> PrintInOption for T where // "where" keyword
    Option<T>: Debug {
    // We want `Option<T>: Debug` as our bound because that is what's
    // being printed. Doing otherwise would be using the wrong bound.
    fn print_in_option(self) {
        println!("{:?}", Some(self));
    }
}

trait Trait {}

impl Trait for i32 {}

// old
fn function1() -> Box<Trait> {
}

// new
fn function2() -> Box<dyn Trait> { // "dyn" keyword
}

fn main() {
    let x = vec![1, 2, 3];

    let equal_to_x = move |z| z == x; // "move" keyword

    println!("can't use x here: {:?}", x);

    let y = vec![1, 2, 3];

    assert!(equal_to_x(y));

    ///

    let c = 'Q';

    // A `ref` borrow on the left side of an assignment is equivalent to
    // an `&` borrow on the right side.
    let ref ref_c1 = c;
    let ref_c2 = &c;

    ////

    let mut num = 5;

    let r1 = &num as *const i32;
    let r2 = &mut num as *mut i32;

    unsafe { // "unsafe" keyword
        println!("r1 is: {}", *r1);
        println!("r2 is: {}", *r2);
    }
}

macro_rules! vec { // custom macro
    ( $( $x:expr ),* ) => {
        {
            let mut temp_vec = Vec::new();
            $(
                temp_vec.push($x);
            )*
            temp_vec
        }
    };
}

macro_rules! foo { // custom macro
    (x => $e:expr) => (println!("mode X: {}", $e)); // <-- println!() is a built in macro
    (y => $e:expr) => (println!("mode Y: {}", $e));
}

vec!() // user macro
foo!(y => 3); // user macro

struct Point {
    x: f64,
    y: f64,
}

// Implementation block, all `Point` methods go in here
impl Point {
    // This is a static method
    // Static methods don't need to be called by an instance
    // These methods are generally used as constructors
    fn origin() -> Point {
        Point { x: 0.0, y: 0.0 }
    }

    // Another static method, taking two arguments:
    fn new(x: f64, y: f64) -> Point {
        Point { x: x, y: y }
    }
}

struct Rectangle {
    p1: Point,
    p2: Point,
}

struct Sheep { naked: bool, name: &'static str }

trait Animal {
    // Static method signature; `Self` refers to the implementor type.
    fn new(name: &'static str) -> Self; // Shows "Self" type keyword.

    // Instance method signatures; these will return a string.
    fn name(&self) -> &'static str;
    fn noise(&self) -> &'static str;

    // Traits can provide default method definitions.
    fn talk(&self) {
        println!("{} says {}", self.name(), self.noise());
    }
}

enum WebEvent {
    // An `enum` may either be `unit-like`,
    PageLoad,
    PageUnload,
    // like tuple structs,
    KeyPress(char),
    Paste(String),
    // or c-like structures.
    Click { x: i64, y: i64 },
}

#[repr(C)]
union MyUnion { // "union" keyword
    f1: u32,
    f2: f32,
}

impl Rectangle {
    // This is an instance method
    // `&self` is sugar for `self: &Self`, where `Self` is the type of the
    // caller object. In this case `Self` = `Rectangle`
    pub fn area(&self) -> f64 {
        // `self` gives access to the struct fields via the dot operator
        let Point { x: x1, y: y1 } = self.p1;
        let Point { x: x2, y: y2 } = self.p2;

        let foo = 123;

        // `abs` is a `f64` method that returns the absolute value of the
        // caller
        ((x1 - x2) * (y1 - y2)).abs()
    }

    fn main() {
        // Increment via closures and functions.
        fn  function            (i: i32) -> i32 { i + 1 }

        // Closures are anonymous, here we are binding them to references
        // Annotation is identical to function annotation but is optional
        // as are the `{}` wrapping the body. These nameless functions
        // are assigned to appropriately named variables.
        let closure_annotated = |i: i32| -> i32 { i + 1 };
        let closure_inferred  = |i     |          i + 1  ;
        let x = a ^ b;
        let x = a & b && c;

        let i = 1;
        // Call the function and closures.
        println!("function: {}", function(i));
        println!("closure_annotated: {}", closure_annotated(i));
        println!("closure_inferred: {}", closure_inferred(i));

        // A closure taking no arguments which returns an `i32`.
        // The return type is inferred.
        let one = || 1;
        let one = && 1;
        let one = ^^ 1;
        println!("closure returning one: {}", one());

    }

    fn foo(x: Box<Fn(i32) -> u32>) {}

    fn perimeter(&self) -> f64 {
        let Point { x: x1, y: y1 } = self.p1; // self keyword
        let mut Point { x: x2, y: y2 } = self.p2; // self keyword
        self::foo() // self keyword
        super::foo() // super keyword

        down_payment = 0.2 * price
        down_payment += 1
        down_payment *= 1
        down_payment -= 1
        down_payment /= 1
        down_payment |= 1
        down_payment *= 1

        let number = 13;
        match number {
            // Match a single value
            1 => println!("One!"),
            // Match several values
            2 | 3 | 5 | 7 | 11 => println!("This is a prime"),
            // Match an inclusive range
            13...19 => println!("A teen"),
            // Handle the rest of cases
            _ => println!("Ain't special"),
        }

        for x in 0..10 {
            println!("{}", x); // x: i32
        }

        loop {
            println!("Loop forever!");
        }

        while true {
            continue;
            break;
            return false;
        }

        if (self.p1 <= self.p2) {
            return
        }
        else {
            return
        }

        if (self.p1 < self.p2) {
            return
        }
        if (self.p1 == self.p2) {
            return
        }
        if (self.p1 >= self.p2) {
            return
        }
        if (self.p1 > self.p2) {
            return
        }
        if (self.p1 != self.p2) {
            return
        }

        2.0 * ((x1 - x2).abs() + (y1 - y2).abs())
        2.0 | ((x1 - x2).abs() + (y1 - y2).abs())
    }

    // This method requires the caller object to be mutable
    // `&mut self` desugars to `self: &mut Self`
    fn translate(&mut self, x: f64, y: f64) {
        self.p1.x /= x;
        self.p2.x *= x;

        self.p1.y += y;
        self.p2.y += y;
    }
}

// `Pair` owns resources: two heap allocated integers
struct Pair(Box<i32>, Box<i32>);

impl Pair {
    // This method "consumes" the resources of the caller object
    // `self` desugars to `self: Self`
    fn destroy(self) {
        // Destructure `self`
        let Pair(first, second) = self;

        println!("Destroying Pair({}, {})", first, second);

        // `first` and `second` go out of scope and get freed
    }
}

const foo = 123

fn main() {
    let a = 1 + foo

    let rectangle = Rectangle {
        // Static methods are called using double colons
        p1: Point::origin(),
        p2: Point::new(3.0, 4.0),
    };

    // Instance methods are called using the dot operator
    // Note that the first argument `&self` is implicitly passed, i.e.
    // `rectangle.perimeter()` === `Rectangle::perimeter(&rectangle)`
    println!("Rectangle perimeter: {}", rectangle.perimeter());
    println!("Rectangle area: {}", rectangle.area());

    let mut square = Rectangle {
        p1: Point::origin(),
        p2: Point::new(1.0, 1.0),
    };

    // Error! `rectangle` is immutable, but this method requires a mutable
    // object
    //rectangle.translate(1.0, 0.0);
    // x ^ Try uncommenting this line

    // Okay! Mutable objects can call mutable methods
    square.translate(1.0, 1.0);

    let pair = Pair(Box::new(1), Box::new(2));

    pair.destroy();

    // Error! Previous `destroy` call "consumed" `pair`
    //pair.destroy();
}

#[macro_use]
extern crate log;

use std::collections::HashMap;
use std::rc::Rc;

mod stuff;

pub enum Flag {
    Good,
    Bad,
    Ugly
}

pub trait Write {
    fn write(&mut self, buf: &[u8]) -> Result<usize>;
}

struct Object<T> {
    flag: Flag,
    fields: HashMap<T, u64>
}

type RcObject<T> = Rc<Object<T>>;

impl<T> Write for Object<T> {
    fn write(&mut self, buf: &[u8]) -> Result<usize> {
        let s = stuff::write_map(&self.fields, buf)?;
        info!("{} byte(s) written", s);
        Ok(s)
    }
}

/* Block comment */
fn main() {
    // A simple integer calculator:
    // `+` or `-` means add or subtract by 1
    // `*` or `/` means multiply or divide by 2
    stuff::AppVersion::print();

    let input = Option::None;
    let program = input.unwrap_or_else(|| "+ + * - /");
    let mut accumulator = 0;

    for token in program.chars() {
        match token {
            '+' => accumulator += 1,
            '-' => accumulator -= 1,
            '*' => accumulator *= 2,
            '/' => accumulator /= 2,
            _ => { /* ignore everything else */ }
        }
    }

    info!("The program \"{}\" calculates the value {}",
             program, accumulator);
}

/// Some documentation `with code`
/// # Heading
/// [Rust](https://www.rust-lang.org/)
#[cfg(target_os="linux")]
unsafe fn a_function<T: 'lifetime>() {
    'label: loop {
        println!("Hello\x20W\u{f3}rld!\u{abcdef}");
    }
}

[Arcitec]

By the way, Rust support may be as easy as this change, which is what I personally added to my settings.json for now:

        "[Monokai Pro]": {
            "textMateRules": [
                {
                    "scope":"storage.type.rust",
                    "settings": {
                        "foreground": "#ff6188"
                    }
                }
            ]
        },

This change colors the Rust struct and impl keywords the same as fn and other keywords.

But will still need to go through the whole language and make sure Monokai Pro really looks nice with all of it... And to consider the macro vs function syntax color as well...

[Arcitec]

Actually... I just compared Rust to JS and Python and there's actually need for a bigger overhaul than I thought. Monokai is supposed to color function, fn, def, let, etc in light blue. And classes in light blue, and functions in green, etc etc. The Rust theme violates all of that by using red for most but not all keywords, etc... It needs a big overhaul to make it consistent with all other languages Monokai actually supports.

[Arcitec]

Here's a revised temporary workaround which colors tokens a bit more consistently. It will be a good start but more work needs to be done, and I can't do any more. I've spent an hour on this so far. I am new to Rust, just starting to learn the language, I don't know all keywords yet (and the syntax highlighting example file above is mostly complete but still lacks some features for sure), and I don't have the energy to go through and try to guess what colors Monokai Pro intends to use in the spirit of the theme. Here's my temporary work:

        "[Monokai Pro]": {
            // TODO: Remove when https://github.com/Monokai/monokai-pro-vscode/issues/160 is fixed.
            "textMateRules": [
                {
                    "scope":"keyword.other.fn.rust,keyword.other.rust,keyword.other.unsafe.rust,keyword.other.where.rust,storage.modifier.const.rust,storage.modifier.dyn.rust,storage.modifier.mut.rust,storage.modifier.static.rust,storage.type.rust,storage.type.core.rust,storage.class.std.rust,support.constant.core.rust,entity.name.type.rust",
                    "settings": {
                        "foreground": "#78dce8"
                    }
                },
                {
                    "scope":"keyword.control.rust,storage.modifier.visibility.rust,keyword.operator.misc.rust",
                    "settings": {
                        "foreground": "#ff6188"
                    }
                },
                {
                    "scope":"meta.type_params.rust", // Important for visual distinction of <> type brackets. Also affects certain type names.
                    "settings": {
                        "foreground": "#fcfcfa"
                    }
                },
                {
                    "scope":"storage.modifier.lifetime.rust,entity.name.lifetime.rust", // Makes "&'static" lifetime modifier red.
                    "settings": {
                        "foreground": "#ff6188"
                    }
                },
                {
                    "scope":"meta.attribute.rust,variable.language.rust", // The "variable.language.rust" is "self" keyword. Note that "super" keyword is "keyword.other.rust" which is used by tons of other keywords.
                    "settings": {
                        "foreground": "#c1c0c0" // Same light gray color as "self" keyword.
                    }
                },
            ]
        },

I went through with JS, Python and CPP files as reference for what colors various keywords should have. One limiting factor is that Rust's syntax highlighter isn't granular enough for true control, so there will be some inconsistencies. And it remains to be seen how these colors feel in actual use.

Also, I've reached the conclusion that I think Rust macros should indeed be the same color as functions.

PS: I used a keyword reference site: https://doc.rust-lang.org/reference/keywords.html ... and painstakingly added real examples for all of those keywords to my syntax highlighting example file in my 1st post. I am not sure this gives 100% language coverage, but it sure as hell will be close to 100%... :-)

[Arcitec]

By the way, even though the syntax rules I ended up with (the previous post above) follow the Monokai style of light blue for functions and types, red for if-else, etc, I still think that Dracula Official has better colors for Rust, because the Monokai result tends to have overly monotone blue lines, whereas Dracula has split things up a bit to get alternating colors, even though that does break consistency in other places. I suggest analyzing what Dracula did, and thinking really hard about what to do with Monokai.

Edit: I did it... have spent a total of two hours tweaking everything now, and have probably gained 100% language coverage now... With these changes, the readability is perfect. So, these are my final colors. They break the Monokai "blue keywords" consistency to gain much better Rust code readability, with the same color patterns as Dracula. The code below is my colors from above PLUS an extra section which overrides some colors back to red instead. You can see for yourself that it's much more readable. I also marked (and commented out) fn and unsafe as potential colors to turn to red, but I really like them as blue for visibility. The comments in the code below explain things...

        "[Monokai Pro]": {
            // TODO: Remove when https://github.com/Monokai/monokai-pro-vscode/issues/160 is fixed.
            "textMateRules": [
                {
                    "scope":"keyword.other.fn.rust,keyword.other.rust,keyword.other.unsafe.rust,keyword.other.where.rust,storage.modifier.const.rust,storage.modifier.dyn.rust,storage.modifier.mut.rust,storage.modifier.static.rust,storage.type.rust,storage.type.core.rust,storage.class.std.rust,support.constant.core.rust,entity.name.type.rust",
                    "settings": {
                        "foreground": "#78dce8"
                    }
                },
                {
                    "scope":"keyword.control.rust,storage.modifier.visibility.rust,keyword.operator.misc.rust",
                    "settings": {
                        "foreground": "#ff6188"
                    }
                },
                {
                    "scope":"meta.type_params.rust", // Important for visual distinction of <> type brackets. Also affects certain type names.
                    "settings": {
                        "foreground": "#fcfcfa"
                    }
                },
                {
                    "scope":"storage.modifier.lifetime.rust,entity.name.lifetime.rust", // Makes "&'static" lifetime modifier red.
                    "settings": {
                        "foreground": "#ff6188"
                    }
                },
                {
                    "scope":"meta.attribute.rust,variable.language.rust", // The "variable.language.rust" is "self" keyword. Note that "super" keyword is "keyword.other.rust" which is used by tons of other keywords.
                    "settings": {
                        "foreground": "#c1c0c0" // Same light gray color as "self" keyword.
                    }
                },

                // Re-adjust some colors to get less Monokai consistency but more contrast a la Dracula theme's Rust highlighting.
                { // Changes all blue colors to red EXCEPT the type-related colors (storage.type.core.rust,storage.class.std.rust,support.constant.core.rust,entity.name.type.rust).
                    "scope":"storage.type.rust,storage.modifier.const.rust,storage.modifier.dyn.rust,storage.modifier.mut.rust,storage.modifier.static.rust,keyword.other.rust,keyword.other.where.rust",
                    "settings": {
                        "foreground": "#ff6188"
                    }
                },
                // These two are commented out because I am leaning towards preferring the readability of having blue "fn" and "unsafe" blocks (same color as types) in a sea of red keywords.
                // {
                //     "scope":"keyword.other.fn.rust", // Not decided yet: Should we force functions to red too, or keep them blue (same color as types)? Keeping them blue is less consistent but makes it easier to see functions.
                //     "settings": {
                //         "foreground": "#ff6188"
                //     }
                // },
                // {
                //     "scope":"keyword.other.unsafe.rust", // Not decided yet: The "unsafe" keyword has the same dilemma. Blue makes unsafe code stand out. Red is more consistent.
                //     "settings": {
                //         "foreground": "#ff6188"
                //     }
                // },
            ]
        },

[Monokai]

Thanks for your work, I've added more rust support to 1.1.15 based on some of your suggestions.