search

slint-ui / slint

Slint is a declarative GUI toolkit to build native user interfaces for Rust, C++, or JavaScript apps.
https://slint.dev
Other
16.85k stars 559 forks source link

Evaluate using Profile-Guided Optimization (PGO) #3909

Open zamazan4ik opened 10 months ago

zamazan4ik commented 10 months ago

Hi!

Recently I checked Profile-Guided Optimization (PGO) improvements on multiple projects. The results are here. E.g. PGO results for LLVM-related tooling are here. According to the tests, PGO usually helps with the compiler and compiler-like workloads (like static analysis or code formatters) - e.g. Clang gets +20% compilation speed with PGO. Since this, I think trying to optimize Slint tools like compiler, code formatted and LSP would be a good idea. I already did some PGO benchmarks on slint-fmt and want to share my results here.

Test environment

Benchmark

For benchmark purposes, I run slint-fmt on all .slint files in the examples directory with slint-fmt <file names>. PGO training phase was done on the same files. Release build is done with cargo build --release --bin slint-fmt. PGO build is done with cargo-pgo (cargo pgo build -- --bin slint-fmt + run on the training workload + cargo pgo optimize build -- --bin slint-fmt).

Benchmarks are done on the same machine, with the same background "noise". The benchmark was performed multiple times - the results are reproducible.

Results

I got the following results (with hyperfine benchmark tool):

hyperfine --warmup 500 --min-runs 5000 `./slint-fmt_release skipped_input_files` `./slint-fmt_optimized skipped_input_files`

Benchmark 1: ./slint-fmt_release skipped_input_files
  Time (mean ± σ):      18.0 ms ±   1.0 ms    [User: 14.1 ms, System: 3.7 ms]
  Range (min … max):    17.3 ms …  36.9 ms    5000 runs

  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.

Benchmark 2: ./slint-fmt_optimized skipped_input_files
  Time (mean ± σ):      15.8 ms ±   0.8 ms    [User: 11.9 ms, System: 3.8 ms]
  Range (min … max):    15.3 ms …  33.6 ms    5000 runs

  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.

Summary
  ./slint-fmt_optimized skipped_input_files ran
    1.14 ± 0.08 times faster than ./slint-fmt_release skipped_input_files

where slint_fmt-release - default Release build, slint-fmt_optimized - Release + PGO-optimized build.

At least in the scenario above PGO helps with achieving better performance with slint-fmt.

Further steps

I can suggest the following action points:

Testing Post-Link Optimization techniques (like LLVM BOLT) would be interesting too (Clang and Rustc already use BOLT as an addition to PGO) but I recommend starting from the usual PGO.

Here are some examples of how PGO optimization is already integrated into other projects:

I am not sure how performance is critical for Slint now. If it isn't the biggest priority right now - could be a nice feature to get in the future.

ogoffart commented 10 months ago

This is a bit hard to do for UI application that needs input from the user because it means that the best way to do pgo is to let the user click through stuff which is not optimal in a build script.

For our tooling like the slint-lsp and the slint-viewer, we could maybe add an option to open file without rendering them, so that will speedup the "compilation" phase, but will not run any of the rendering code which may also be important.