Open zamazan4ik opened 1 week ago
Hey, thanks for sharing! 10-15% improvement is indeed a lot so worth exploring. The microbenchmarks are currently all we have, but once we get TPC-H benchmarks going (#4), worth checking them out too. Really curious to hear more about what optimizations PGO does. I opened discussions for this repository so we can continue there.
Hi!
Thank you for the project! I evaluated Profile-Guided Optimization (PGO) on many projects - all the results are available at https://github.com/zamazan4ik/awesome-pgo . Since this compiler optimization works well in many places including optimization databases (including SQLite), I decided to apply it to the project - here are my benchmark results.
Test environment
limbo
version:main
branch on commit93a634d334a6a0629f516e31691c88742519ceed
Benchmark
For benchmark purposes, I use built-in into the project benchmarks. For PGO optimization I use cargo-pgo tool. Release bench result I got with
taskset -c 0 cargo bench
command. The PGO training phase is done withtaskset -c 0 cargo pgo bench
, PGO optimization phase - withtaskset -c 0 cargo pgo optimize bench
.taskset -c 0
is used for reducing the OS scheduler influence on the results. All measurements are done on the same machine, with the same background "noise" (as much as I can guarantee).Results
I got the following results:
According to the results, PGO measurably improves Limbo's performance.
Rusqlite performance wasn't improved since
cargo-pgo
cannot optimize non-Rust code with PGO. It's possible to achieve with manually passing corresponding compiler switches to the C-part but I didn't do that during this test since I was interested only in optimizing Limbo's speed.Further steps
I can suggest the following action points:
I would be happy to answer your questions about PGO.
P.S. It's just a benchmark report with some an idea for improvement for the project. I created the Issue only because Discussions are disabled for the repository.