Classes 2.0

[divmain]

• https://github.com/neon-bindings/neon/issues/19
• https://github.com/neon-bindings/neon/issues/62

[dherman]

So the idea in general here is to have a coercion protocol (via traits) and to allow method definitions in classes to have type annotations that use the coercion protocol to abstract away all the conversion boilerplate. Moreover we should make it possible to define functions and modules declaratively with the macro syntax. So at that point we should have a nice "neon mullet" (JS business up front, Rust party in the back). Quick minimal example:

module! {
    export function foo(x: u32) -> u32 {
        Ok(x + 1)
    }
}

[nixpulvis]

I'd probably use the word pub instead of export since we're writing Rust inside module!.

[dherman]

I'd probably use the word pub

I disagree: the idea is that the outside syntax is JavaScript, and the type syntax and function body syntax are Rust, because it's presenting a JS interface to the outside world while using a Rust implementation (including automatic conversions to Rust types) internally.

[matklad]

Some field experience with classes 1.0 (from https://github.com/matklad/fall/). Although the tone will sound slightly negative, I mightily admire the work done, which allows to bind Rust and nodejs at all! :) ❣️

Using macro DSL is quite hard: it is opaque, you have to do contortions to impl classes for generic types with < in their names, and the only way to abstract over this macro is with another macro.

Here's how I try to reuse some functionality to declare similar classes: https://github.com/matklad/fall/blob/3d099c94cd3b25ca93e32d39851aaeefab3799ae/code/generic_backend/src/lib.rs#L52-L87. This felt bad to write.

What's more, this DSL forces huge swaths of Rust code to be inside of macro invocation, which kills a lot of IDE features (of course the IDEs are not mature yet, but it always would be true that the code inside the macro will be harder for IDEs to understand). It also effectively invents (ad-hoc, informally-specified, bug-ridden, slow implementation of) a new language withing Rust itself. the method "keyword" definitely feels alien.

There are also problems on the client side. I write my client code in TypeScript, and, naturally, I don't have any typechecks when calling Rust classes.

So, with all that, I find that the best pattern for my use-case was actually C-style OO:

On the Rust side, I declare a dumb class which is just a wrapper so that I can return a Rust type to JavaScript. I also declare a factory function and a bunch of "methods": functions, that take a wrapper as a first argument.

On the JS side, I wrap a pair of (my backend neon module, wrapper class) into a proper TypeScript class, which gives me typed wrapper and allows me to inspect, verify and massage arguments before they are handled to Rust.

So, for me personally, I would rather prefer https://github.com/neon-bindings/neon/issues/286 (a generic JsBox class to send data to JS) to be fixed, rather then a more elaborate macro-based DSL.

PS. Have I already told you that neon is just awesome? :)

[dherman]

@matklad Have I told you your feedback on neon is just awesome? ;)

Although the tone will sound slightly negative, I mightily admire the work done, which allows to bind Rust and nodejs at all! :) ❣️

Please don’t worry—negative experience reports are extremely valuable! But I appreciate your kind words as well.

Using macro DSL is quite hard: it is opaque, you have to do contortions to impl classes for generic types with < in their names,

That part is basically a bug that should be fixed. Unless we run into some fundamental parsing limitation, I think it’s little more than an oversight on my part.

and the only way to abstract over this macro is with another macro

That’s definitely true.

What's more, this DSL forces huge swaths of Rust code to be inside of macro invocation, which kills a lot of IDE features (of course the IDEs are not mature yet, but it always would be true that the code inside the macro will be harder for IDEs to understand).

Yes, this is also true. Although it doesn’t have to be—I would like to see Rust push on having an “expand-for-IDE” API for macros, so they can do fast, side-effect-free expansions that may not generate correct code for executing at runtime but they can provide sufficient information for IDEs to understand the code. I’ve spoken with some core team members who are interested in this but it’s of course not something that can happen overnight. So for the meantime you’re absolutely right that the macro will sometimes confuse IDEs.

It also effectively invents (ad-hoc, informally-specified, bug-ridden, slow implementation of) a new language withing Rust itself. the method "keyword" definitely feels alien.

“Slow” is a bit of a stretch but hey, you got your Greenspun’s law reference in. ;P Seriously I don’t like method either and I wonder if we can eliminate it.

There are also problems on the client side. I write my client code in TypeScript, and, naturally, I don't have any typechecks when calling Rust classes.

Yeah, I’ve been wondering if there’s some IDL-like approach we could use to generate .d.ts type declarations.

On the JS side, I wrap a pair of (my backend neon module, wrapper class) into a proper TypeScript class, which gives me typed wrapper and allows me to inspect, verify and massage arguments before they are handled to Rust.

There’s definitely a lot of sense to doing the “polish” work of a class (eg constructor and method APIs, massaging arguments, etc) in JS instead of Rust.

So, for me personally, I would rather prefer neon-bindings/neon#286 (a generic JsBox class to send data to JS) to be fixed, rather then a more elaborate macro-based DSL.

I completely agree this is worth building, to see how far we can push it.

And I understand and largely agree with a lot of your concerns about macros.

At the same time, I’m not ready to give up on the macro DSL. In part this is because I think we can eliminate a lot of the boilerplate around argument parsing and conversion.

And in part it’s because I want to build as gentle an on-ramp for JS programmers just trying Rust for the first time. Yes, you can argue it’s not “real Rust” to use a macro DSL, but to a first time Rust programmer it makes no difference: they just want to get their “hello world” working. Something that looks like TypeScript syntax (function/method declarations with type annotations) is familiar enough to conceptually pattern match without really understanding.

That said, I think we just need to try both something like the simpler primitive you referenced and the macro DSL, and see what works best in what environments. I don’t see a reason we can’t explore both.

And we might find a less elaborate macro, like just a function macro with automatic argument parsing, in conjunction with styles like the one you describe, end up working best, and a class macro ends up being more trouble than it’s worth. You’ve definitely given me a good alternative mental model to chew on.

PS. Have I already told you that neon is just awesome? :)

💕

[matklad]

Although it doesn’t have to be—I would like to see Rust push on having an “expand-for-IDE” API for macros, so they can do fast, side-effect-free expansions that may not generate correct code for executing at runtime but they can provide sufficient information for IDEs to understand the code. I’ve spoken with some core team members who are interested in this but it’s of course not something that can happen overnight.

As an IDE writer myself, I can assure you that code inside of macro invocations will always be more annoying to write than the usual code :) There are two reasons for this (even if we assume support from compiler): performance and robustness.

Great IDEs make use of a true syntax tree when reacting to user's keystrokes. You need the tree to correctly indent lines after \n, add or remove trailing commas, glue or split string literals over multiple lines, that sort of stuff. That means that getting a syntax tree is a part of typing latency of at least certain characters. Usually, this is not a problem, because constructing a syntax tree is a very local operation: you need to know only about the current file, you can always relex it incrementally, and often times you can even reuse parsing results. However, with macros you can't parse code using purely local information: the code inside macros is formally a token tree, and to get a real parse tree you must resolve the macro to its definition, and it is necessary a global operation (I.e, you need to look at other files, consult caches, etc).

Also it can happen that you can't resolve macro because the build is broken: internet is down and Cargo is acting up, or you are resolving a merge conflict and <<<<<< break everything, or you are editing a file but have not linked it to the rest of project by the mod foo; declaration, or you are editing a temporary in-memory buffer. In all of these cases, you simply can't resolve macros, and so IDE is forced to look at the code as a token tree, even if it looks like a Rust function to a human :)

Obviously, I am not trying to say that one should not use macros at all because IDEs are stupid, and I don't even try to say that macros will be too annoying for IDEs. They'll be somewhat more annoying :)

In part this is because I think we can eliminate a lot of the boilerplate around argument parsing and conversion.

It would be great to have some non-macro based solution for this as well. I would love to have something like

let (some_int, some_str, callback, data, obj): (i32, String, JsFunction, MySerializableData, MyJsClass)
    = parse_args(call.scope, call.arguments)?;

Currently I am using neon-serde for something similar, but it works only for serializable objects, and if I have a mixture of POD data and, say, js callbacks, I have to do manual unwrapping of arguments.

And in part it’s because I want to build as gentle an on-ramp for JS programmers just trying Rust for the first time. Yes, you can argue it’s not “real Rust” to use a macro DSL, but to a first time Rust programmer it makes no difference: they just want to get their “hello world” working. Something that looks like TypeScript syntax (function/method declarations with type annotations) is familiar enough to conceptually pattern match without really understanding.

Yeah, that's a very valid use-case for DSLs! Two thoughts on this topic:

1) For "hello world" use case, it's really important to emphasize functions and not classes, because the are simpler.

2) Marcos for beginners are a double edged sword. On the one hand, you can map rust code to a form, which resembles what you would have written in JavaScript. On the other hand, if you make an error when using macro, the error message won't be great.