nvzqz / divan

Fast and simple benchmarking for Rust projects
https://nikolaivazquez.com/blog/divan/
Apache License 2.0
849 stars 24 forks source link

`cargo flamegraph` support #38

Open lpotthast opened 7 months ago

lpotthast commented 7 months ago

Using https://github.com/flamegraph-rs/flamegraph with

cargo flamegraph --bench benchmarks

with benches/benchmarks.rs being present and defined in Cargo.toml leads to benchmark names being printed but no benchmark-code being actually executed.

It would be great if there was support between divan and flamegraph! If I do something wrong and this should already work, please let me know.

cbarrick commented 4 months ago

In my experience, it seems like the benchmarks are being run, but only a single sample of each. I can verify this be having a single benchmark that is expected to run for a nontrivial amount of time (like 20s).

Using cargo bench with divan, I know how long a single sample should take, and using cargo flamegraph appears to take that exact amount of time. The resulting flamegraph looks plausible.

However, no measurements are printed to the console, and --sample-count is ignored.

Maybe this is WAI and we just need some documentation?

cbarrick commented 4 months ago

I did some debugging on this.

It turns out that cargo bench --bench=foo will call you benchmark like:

./target/bench/deps/foo --bench

But cargo flamegraph --bench=foo will call your benchmark like:

./target/bench/deps/foo

i.e. cargo flamegraph does not pass the --bench flag. Without this flag, divan::main will run your benchmarks once without reporting stats.

The quickfix is to invoke your benchmark like:

cargo flamegraph --bench=foo -- --bench

I am not sure if this is a bug in cargo-flamegraph, a bug in divan, or working as intended.