Recently I tested Profile-Guided Optimization (PGO) compiler optimization on different projects in different software domains - all the results are available at https://github.com/zamazan4ik/awesome-pgo . Since PGO shows measurable improvements in many cases, I decided to perform PGO benchmarks on this library too. Here are my results - I hope they will be helpful for someone.
Test environment
Fedora 39
Linux kernel 6.8.9
AMD Ryzen 9 5900x
48 Gib RAM
SSD Samsung 980 Pro 2 Tib
Compiler - Rustc 1.78.0
nom version: the latest for now from the main branch on commit 36e74ab8991ae1656edc91c86d4386cca73e5a50
Disabled Turbo boost
Benchmark
For benchmark purposes, I use built-in into the project benchmarks. For PGO optimization I use cargo-pgo tool. Release bench result I got with taskset -c 0 cargo bench command. The PGO training phase is done with taskset -c 0 cargo pgo bench, PGO optimization phase - with taskset -c 0 cargo pgo optimize bench. taskset -c 0 is used for reducing the OS scheduler influence on the results. All measurements are done on the same machine, with the same background "noise" (as much as I can guarantee).
According to the results, PGO measurably improves the library's performance in many cases. I expected such or similar results since PGO is especially good for things like parsers.
Further steps
I can suggest the following action points:
Perform more PGO benchmarks with other datasets (if you are interested enough). If it shows improvements - add a note to the documentation (the README file?) about possible improvements in the library's performance with PGO.
Probably, you can try to get some insights about how the code can be optimized further based on the changes that the compiler performed with PGO. It can be done via analyzing flamegraphs before and after applying PGO to understand the difference.
Testing Post-Link Optimization techniques (like LLVM BOLT) would be interesting too (Clang and Rustc already use BOLT as an addition to PGO). However, I recommend starting from the usual PGO since it's a much more stable technology with fewer limitations.
I would be happy to answer your questions about PGO.
P.S. Please do not treat the issue like a bug or something like that. Since the "Discussions" functionality is disabled in this repo, I created the Issue instead.
Hi!
Recently I tested Profile-Guided Optimization (PGO) compiler optimization on different projects in different software domains - all the results are available at https://github.com/zamazan4ik/awesome-pgo . Since PGO shows measurable improvements in many cases, I decided to perform PGO benchmarks on this library too. Here are my results - I hope they will be helpful for someone.
Test environment
nomversion: the latest for now from themainbranch on commit36e74ab8991ae1656edc91c86d4386cca73e5a50Benchmark
For benchmark purposes, I use built-in into the project benchmarks. For PGO optimization I use cargo-pgo tool. Release bench result I got with
taskset -c 0 cargo benchcommand. The PGO training phase is done withtaskset -c 0 cargo pgo bench, PGO optimization phase - withtaskset -c 0 cargo pgo optimize bench.taskset -c 0is used for reducing the OS scheduler influence on the results. All measurements are done on the same machine, with the same background "noise" (as much as I can guarantee).Results
I got the following results:
According to the results, PGO measurably improves the library's performance in many cases. I expected such or similar results since PGO is especially good for things like parsers.
Further steps
I can suggest the following action points:
Testing Post-Link Optimization techniques (like LLVM BOLT) would be interesting too (Clang and Rustc already use BOLT as an addition to PGO). However, I recommend starting from the usual PGO since it's a much more stable technology with fewer limitations.
I would be happy to answer your questions about PGO.
P.S. Please do not treat the issue like a bug or something like that. Since the "Discussions" functionality is disabled in this repo, I created the Issue instead.