Venial is a small parser for Rust proc macros.
When writing proc macros that need to parse Rust code (such as attribute and derive macros), the most common solution is to use the syn crate. Syn can parse arbitrary valid Rust code, and even Rust-based DSLs, and return versatile data structures that can be inspected and mutated in powerful ways.
It's also extremely heavy. In one analysis of lqd's early 2022 benchmark collection, the author estimates that syn is reponsible for 8% of compile times of the benchmark, which accounts for Rust's most popular crates. There are subtleties (eg this isn't necessarily critical path time, but syn is often in the critical path anyway), but the overall takeaway is clear: syn is expensive.
And yet, a lot of the power of syn is often unneeded. If we look at the crates that depend on syn, we can see that the 5 most downloaded are:
Of these, proc-macro-hack is deprecated, and the other four only need to parse basic information on a type.
Other popular reverse-dependencies of syn (such as futures-macro, tokios-macros, async-trait, etc) do use syn's more advanced features, but there's still room for a lightweight parser in proc-macros.
Venial is that parser.
Venial is extremely simple. Most of its implementation is in the parse.rs
file, which is about 350 lines at the time I'm writing this README. This is because the Rust language has a very clean syntax, especially for type declarations.
Venial has no dependency besides proc-macro2 and quote.
To achieve this simplicity, venial makes several trade-offs:
struct MyStruct {}
). It can't parse expressions or statements. For now, only types and functions are supported.foo_bar: &mut Foo<Bar, dyn Foobariser>
, venial will dutifully give you this type as a sequence of tokens and let you interpret it.Note though that venial will accept any syntactically valid declaration, even if it isn't semantically valid. The rule of thumb is "if it compiles under a #[cfg(FALSE)]
, venial will parse it without panicking".
The only exception is enum discriminants. Venial only supports enum discriminants with a single token, or a token-group. Eg:
enum MyEnum {
A = 42, // Ok
B = "hello", // Ok
C = CONSTANT, // Ok
D = FOO + BAR, // MACRO ERROR
E = (FOO + BAR), // Ok
}
This is because parsing complex discriminants requires arbitrary expression parsing, which is beyond the scope of this crate.
(Note: venial currently panics on unsupported declarations, eg traits, aliases, etc. Also, function support is incomplete.)
use venial::{parse_declaration, Declaration};
use quote::quote;
let enum_type = parse_declaration(quote!(
enum Shape {
Square(Square),
Circle(Circle),
Triangle(Triangle),
}
));
let enum_type = match enum_type {
Declaration::Enum(enum_type) => enum_type,
_ => unreachable!(),
};
assert_eq!(enum_type.variants[0].0.name, "Square");
assert_eq!(enum_type.variants[1].0.name, "Circle");
assert_eq!(enum_type.variants[2].0.name, "Triangle");
I haven't performed any kind of formal benchmark yet. That said, I compared this fork of miniserde using venial to the equivalent miniserde commit, and got the following results:
$ cargo check -j1 # miniserde-venial, clean build
Finished dev [unoptimized + debuginfo] target(s) in 6.30s
$ cargo check -j1 # miniserde, clean build
Finished dev [unoptimized + debuginfo] target(s) in 9.52s
$ cargo check -j4 # miniserde-venial, clean build
Finished dev [unoptimized + debuginfo] target(s) in 3.17s
$ cargo check -j4 # miniserde, clean build
Finished dev [unoptimized + debuginfo] target(s) in 4.79s
My machine is desktop computer with an AMD Ryzen 7 1800x (8 cores, 16 threads), I have 32GB of RAM and a 2.5TB SSD.
As we can see, using venial instead of syn shaves about 3.2s off total build times in single-threaded builds, and 1.6s in 4-threaded builds.
Most of the difference comes from syn and venial themselves: cargo check --timings
shows that syn takes 2.11s to compile and venial takes 0.58s in 4-threaded builds.
I'm not showing codegen builds, release mode builds, 16-threads builds and the like, but the trend stays roughly the same: for the miniserde project, switching to venial removes ~30% of the build time.
That's a fairly complicated question to answer. Two years after starting the project, my answer is now "Only for niche use-cases".
If you take the most optimistic interpretation, these results are great! On a single-threaded machine, switching shaves three seconds off, a whole third of the build time!
In reality, there are a lot of complicating factors:
All in all, it's questionable whether the benefits are worth porting your derive crate from syn to venial (though my experience so far has been that porting isn't that hard).
Another thing to keep in mind is that this is a very young library. There has been very little effort to optimize it or profile it so far, and further versions may give a better build time reduction.
tl;dr: You can probably shave off a few seconds off your clean builds with venial. Incremental builds see no benefits.
Pull requests are welcome.
I have no intention to work on venial in the near future myself, but I will still merge PRs.
Some possible improvements: