Evaluate using Profile-Guided Optimization (PGO) for optimizing Root's performance

zamazan4ik commented 5 months ago

Explain what you would like to see improved and how.

I checked various compiler optimizations like Profile-Guided Optimization (PGO) on many projects - all the results are available at https://github.com/zamazan4ik/awesome-pgo . Since such optimizations help with optimizing CPU-intensive apps, I think it can be an interesting idea to optimize the Root project with PGO since the project cares a lot about performance.

I can suggest the following ideas:

Perform PGO benchmarks with some typical for the project workloads. If it shows improvements - add a note to the documentation about possible improvements in Root's performance with PGO. E.g. for Cling I already proposed evaluating PGO: https://github.com/root-project/cling/issues/508
Perform testing Post-Link Optimization techniques (like LLVM BOLT) would be interesting too (Clang and Rustc already use BOLT as an addition to PGO). However, I recommend starting with the usual PGO since it's a much more stable technology with fewer limitations.

ROOT version

Latest

Installation method

Any

Operating system

Any

Additional context

No response

dpiparo commented 5 months ago

Hi, thanks for this proposal. PGO has been investigated in detail by some HEP experiments, e.g. CMS https://indico.cern.ch/event/1106990/contributions/4991214/ . There was no symbol coming from ROOT which was really sticking out. At the moment don' see clearly how a general purpose framework such as ROOT could benefit from PGO, given the variety of workflows that use ROOT - from HLT to Reco, to Simulation and analysis - and all in different experiments, with very different software stacks.

Do you have something in particular in mind? Can you invest some time in the topic to prove your point? It would be interesting for us to comment on concrete measurements.

zamazan4ik commented 5 months ago

PGO has been investigated in detail by some HEP experiments, e.g. CMS https://indico.cern.ch/event/1106990/contributions/4991214/ . There was no symbol coming from ROOT which was really sticking out.

Thanks a lot for sharing the results!

At the moment don' see clearly how a general purpose framework such as ROOT could benefit from PGO, given the variety of workflows that use ROOT - from HLT to Reco, to Simulation and analysis - and all in different experiments, with very different software stacks.

I agree that various workloads can benefit from PGO differently. What we can do is try to measure PGO performance improvements in multiple scenarios (like all the scenarios you mentioned above), and post the results somewhere (e.g. here). If the performance improvement is measurable for some scenarios - put this information into the project's documentation. In this case, users will be aware about additional ways to optimize Root.

Do you have something in particular in mind? Can you invest some time in the topic to prove your point? It would be interesting for us to comment on concrete measurements.

Actually, I don't have in mind any specific workload since I don't have enough experience with Root (at least yet). Later maybe I will be able to spend some time on an experiment with PGO and Root. However, my TODO list for such experiments is a bit huge so I cannot promise any results soon.

dpiparo commented 5 months ago

At the moment we have no evidence of potential PGO optimizations ROOT could provide, also given the contribution cited above (a paper is coming soon): no symbol in the ROOT libraries is appearing in the list of the ones that benefit from PGO. While I agree that it could be interesting to expand the findings, we do not have substantial effort to invest there at the moment. I propose to close this issue now and warmly invite you to re-open it as soon as you have concrete results to share and that can be transformed in a list of concrete actions to take on our side.

zamazan4ik commented 5 months ago

Why not just leave the issue open? If anyone from the community finds the issue interesting - they can start working on it. However, I kindly propose enabling the "Discussion" functionality for the repository and then moving the issue to the "Discussions" under the "Ideas" built-in label. Having the issue closed is not clear for newcomers since they at first look at open issues to start work at, not closed issues.