rust-lang / rust-analyzer

A Rust compiler front-end for IDEs
https://rust-analyzer.github.io/
Apache License 2.0
13.8k stars 1.53k forks source link

Not showing "run benchmark" for non-std benchmarks #17274

Open alshdavid opened 1 month ago

alshdavid commented 1 month ago

Repro

When using the unstable standard library benchmark, I am prompted to run a specific benchmark.

Using a non std benchmarker does not work. Marking a benchmark with #[bench] or #[divan::bench] results in no "run benchmark" prompt appearing when I am not using the built-in unstable bench.

image

Rust Analyzer Version v0.3.1958 Rust Version 1.78.0 Vscode Version 1.89.1

nvzqz commented 1 week ago

It would be great if Rust Analyzer could run individual Divan benchmarks, so let me know what's needed on my end in Divan to support this. I think to support this today, we can invoke cargo bench -- foo_bench::benchmark to run the specific benchmark.

Veykril commented 1 week ago

For context how this works with custom #[test] attributes is that all of them usually end up generating an original #[test] attribute at some point. Obviously this isn't the case for custom benchers, since #[bench] itself is unstable. I think the ideal way would be to have a tool attribute like #[rust_analyzer::runnable(something here that tells r-a how to construct the runnable)] (https://github.com/rust-lang/rust-analyzer/issues/11556). That has downsides wrt to MSRV though given the tool attribute for r-a only made it into nightly now. We can probably just do heuristics here for the time being for general #[bench] attributes checking syntactically (I assume most customer bencher are run the same way?).

What does the bench attribute here expand to roughly?

nvzqz commented 1 week ago

I assume most customer bencher are run the same way?

My understanding is that Divan is the only one using a #[bench] attribute. The other popular benchmarking harnesses are Criterion and Tango, which do not register benchmarks with attribute macros.

What does the bench attribute here expand to roughly?

Given the following ways to use the attribute in the same file location as the original post:

#[divan::bench]
fn example1() {}

#[divan::bench]
fn example2(b: Bencher) {}
These expand to: ```rs static __DIVAN_BENCH_EXAMPLE1: ::divan::__private::BenchEntry = { { #[used] #[link_section = "__DATA,__mod_init_func,mod_init_funcs"] static PUSH: extern "C" fn() = push; extern "C" fn push() { static NODE: ::divan::__private::EntryList< ::divan::__private::BenchEntry, > = ::divan::__private::EntryList::new(&__DIVAN_BENCH_EXAMPLE1); ::divan::__private::BENCH_ENTRIES.push(&NODE); } } ::divan::__private::BenchEntry { meta: ::divan::__private::EntryMeta { raw_name: "example1", display_name: "example1", module_path: "foo_crate::foo::foo_bench", location: ::divan::__private::EntryLocation { file: "src/foo/foo_bench.rs", line: 3u32, col: 1u32, }, get_bench_options: ::divan::__private::None, cached_bench_options: ::divan::__private::OnceLock::new(), }, bench: ::divan::__private::BenchEntryRunner::Plain(|divan| { divan.bench(example1) }), } }; static __DIVAN_BENCH_EXAMPLE2: ::divan::__private::BenchEntry = { { #[used] #[link_section = "__DATA,__mod_init_func,mod_init_funcs"] static PUSH: extern "C" fn() = push; extern "C" fn push() { static NODE: ::divan::__private::EntryList< ::divan::__private::BenchEntry, > = ::divan::__private::EntryList::new(&__DIVAN_BENCH_EXAMPLE2); ::divan::__private::BENCH_ENTRIES.push(&NODE); } } ::divan::__private::BenchEntry { meta: ::divan::__private::EntryMeta { raw_name: "example2", display_name: "example2", module_path: "foo_crate::foo::foo_bench", location: ::divan::__private::EntryLocation { file: "src/foo/foo_bench.rs", line: 6u32, col: 1u32, }, get_bench_options: ::divan::__private::None, cached_bench_options: ::divan::__private::OnceLock::new(), }, bench: ::divan::__private::BenchEntryRunner::Plain(example2), } }; ```

[!NOTE] The two benchmarks only functionally differ in the BenchEntry.bench property, and the only platform specific code is link_section.