search

rust-ml / linfa

A Rust machine learning framework.
Apache License 2.0
3.76k stars 249 forks source link

Plotting in Criterion benchmarks are extremely slow #140

Closed YuhanLiin closed 3 years ago

YuhanLiin commented 3 years ago

Many of our Criterion benchmarks (particularly the ones in linfa-clustering) call plot_config, which causes the runner to create plots at the end of the benchmarks. The issue is that the plot creation takes seemingly forever. I'm in favour of removing the plots since I don't really use them, but if someone objects to that then we can look into it.

bytesnake commented 3 years ago

According to criterion's documentation the default is gnuplot and it falls back to the plotter crate if the binary is not existing. I would suspect that gnuplot is fast enough, therefore could the issue just be that we have to install it in the runner before executing the actions? https://bheisler.github.io/criterion.rs/book/user_guide/plots_and_graphs.html Otherwise let's disable them by passing --noplot, i.e. cargo bench -- --noplot

YuhanLiin commented 3 years ago

The criterion benchmarks don't run in CI (for good reason). The slowness is just on my side, so I'll try with gnuplot

YuhanLiin commented 3 years ago

Everything works fine with gnuplot. It seems that there's something buggy going on with the plotter backend, as seen here: https://github.com/bheisler/criterion.rs/issues/471. Since we can't turn off plotter we should probably mention this in the README somewhere

YuhanLiin commented 3 years ago

Best to keep this one open until the plotters backend actually gets fixed (or removed).