search

radareorg / radare2

UNIX-like reverse engineering framework and command-line toolset
https://www.radare.org/
GNU Lesser General Public License v3.0
20.75k stars 3.01k forks source link

Evaluate Profile-Guided Optimization (PGO) profits on Radare2 #22032

Open zamazan4ik opened 1 year ago

zamazan4ik commented 1 year ago

Description

Profile-Guided Optimization (PGO) optimization technique can help with improving CPU performance. Here results from applying PGO on different open-source projects in various domains.

I think it would be interesting to see PGO results on Radare2 as well. If you have a suggestion for a good benchmark to test it - please let me know. If you already have some results about PGO with Radare2 - please share them here! If PGO will show some improvements on Radare2 - will be great to see a note about PGO in Radare2 documentation.

trufae commented 1 year ago

PGO is an interesting optimization, but i have never benchmarked or tested it, but seems like r2 could be a good target for testing this technique. But right now my main focus is on implementing optimizations in the code rather than at compilation stage. i can say that r2 is about -O2 is 2-3x faster than -O1, and LTO helps but cant say numbers about pgo rn

trufae commented 1 year ago

I added a make pgo target in SDB and followed the lines described in https://llvm.org/devmtg/2020-09/slides/PGO_Instrumentation.pdf and the results are quite dissapointing, i guess the code can't be optimized much more but this didnt bring more than 1% perf improvement

trufae commented 1 year ago

Also, the profile data is compiler-specific, so needs to be generated in the CI while running the testsuite.

zamazan4ik commented 1 year ago

I added a make pgo target in SDB and followed the lines described in https://llvm.org/devmtg/2020-09/slides/PGO_Instrumentation.pdf

Did you use -fprofile-instr-generate/-fprofile-instr-use pair of commands? Are these flags propagated to the radare2 dependencies too (PGOing dependencies with the program helps more)? On which workload did you collect the runtime profile? Usually, the test suite is not a good enough source of the profile since the test suite tries to execute all paths, not only happy real-life ways (but I may be wrong here since I know nothing about radare2 test suite).

Also, the profile data is compiler-specific, so needs to be generated in the CI while running the test suite.

Yes, they are compiler-specific. But you can collect profile once, then commit it and re-generate it only when you change a compiler and/or compiler version. But better way always is to commit somehow a sample workload instead of ready to use perf counters.