search

project-machine / puzzlefs

Apache License 2.0
378 stars 18 forks source link

Evaluate Profile-Guided Optimization (PGO) and LLVM BOLT #113

Open zamazan4ik opened 9 months ago

zamazan4ik commented 9 months ago

Hi!

Recently I checked Profile-Guided Optimization (PGO) improvements on multiple projects. The results are here. Since PGO shows improvements for many workloads, I think trying to optimize PuzzleFS with PGO can be a good idea. E.g. compression routines can be optimized with PGO since the official Zstd implementation supports PGO build.

I can suggest the following action points:

Here you can find examples of how PGO (and sometimes LLVM BOLT) is already integrated into different projects. Here are documentation examples with PGO in different projects.

Maybe testing Post-Link Optimization techniques (like LLVM BOLT) would be interesting too but I recommend starting from the usual PGO.

For the Rust projects, I suggest PGO optimizing with cargo-pgo (it supports LLVM BOLT too).

ariel-miculas commented 9 months ago

I'm wondering whether the zstd bindings should also support PGO or if this is not a necessary step for optimizing the Zstd compression.

For the Rust projects, I suggest PGO optimizing with cargo-pgo (it supports LLVM BOLT too).

Do you have an example of a Rust project that has support for PGO? I'm referring to the build option you've mentioned when you suggested to provide an easier way for end-users to optimize PuzzleFS.

zamazan4ik commented 9 months ago

I'm wondering whether the zstd bindings should also support PGO or if this is not a necessary step for optimizing the Zstd compression.

Since it's the binding library - yes, it needs special support. More discussions about that topic can be found here: https://github.com/Kobzol/cargo-pgo/issues/38 . But it's true only for non-Rust dependencies - Rust dependencies and PuzzleFS own code can be optimized with PGO.

Do you have an example of a Rust project that has support for PGO? I'm referring to the build option you've mentioned when you suggested providing an easier way for end-users to optimize PuzzleFS.

Sure, I have some examples:

Maybe examples from the non-Rust projects would be helpful too (since there is no huge difference in PGO implementation details between Rust and C/C++ projects):