search

dora-rs / dora

DORA (Dataflow-Oriented Robotic Application) is middleware designed to streamline and simplify the creation of AI-based robotic applications. It offers low latency, composable, and distributed dataflow capabilities. Applications are modeled as directed graphs, also referred to as pipelines.
https://dora-rs.ai
Apache License 2.0
1.35k stars 67 forks source link

Evalaute Profile-Guided Optimization (PGO) and LLVM BOLT #331

Open zamazan4ik opened 11 months ago

zamazan4ik commented 11 months ago

Hi!

Recently I checked Profile-Guided Optimization (PGO) improvements on multiple projects. The results are here.

Since PGO showed measurable improvements in many loads (including many databases) I think it could be helpful to check PGO on Dora.

We need to perform PGO benchmarks on Dora. And if it shows improvements in throughput/latency/CPU usage/smth else - add a note about possible improvements Dora's performance with PGO. Providing an easier way (e.g. a build option) to build scripts with PGO can be useful for the end-users too.

For the Rust projects, I recommend starting with https://github.com/Kobzol/cargo-pgo . After PGO I can suggest you apply LLVM BOLT to the project.

phil-opp commented 11 months ago

Thanks for the suggestion! This seems like a good idea in general. Right now we have other things with higher priority, but I put it in our backlog. (If someone else wants to look into this, we're of course happy to merge a PR.)

zamazan4ik commented 2 months ago

@haixuanTao do you have PGO benchmarks for Dora to share with the community? Just curious since you closed the issue as "completed".

haixuanTao commented 2 months ago

I haven't but we haven't really done in it close to 10month so might as well close it to focus on our priority.

Not an expert on PGO, how shall we try to benchmark dora?

zamazan4ik commented 2 months ago

I haven't but we haven't really done in it close to 10month so might as well close it to focus on our priority.

However, when you close the issue it's a signal to the community that you are not interested in this activity. If you don't want to spend your time with the issue - it's fine! Just attach a label to the issue smth like "help wanted" and leave the issue open.

Not an expert on PGO, how shall we try to benchmark dora?

I would suggest to start in the following way:

  1. Think about what is the typical workload for the project and how you can measure performance for it (CPU usage, time to complete the workload - something like that)
  2. Recompile dora with instrumentation PGO: https://doc.rust-lang.org/rustc/profile-guided-optimization.html
  3. Train it on your typical workload from step 1
  4. Recompile dora once again with the collected PGO profiles
  5. Compare performance between a regular Release build and a Release + PGO optimized build