search

risingwavelabs / risingwave

Best-in-class stream processing, analytics, and management. Perform continuous analytics, or build event-driven applications, real-time ETL pipelines, and feature stores in minutes. Unified streaming and batch. PostgreSQL compatible.
https://go.risingwave.com/slack
Apache License 2.0
6.76k stars 557 forks source link

Evaluate Profile-Guided Optimization (PGO) #11859

Open zamazan4ik opened 1 year ago

zamazan4ik commented 1 year ago

Is your feature request related to a problem? Please describe.

Not exactly a problem - just an idea of how to improve the performance of Risingwave.

Describe the solution you'd like

There are a lot of examples of different software, where Profile-Guided Optimization (PGO) helps a lot with performance - you can check it here. E.g. in this list are a lot of databases like Redis, MongoDB, and PostgreSQL.

We need to check the PGO effects on Risingwave, and if it works well - write a note in the Risingwave documentation about PGO (like it's already done for ClickHouse). I'd appreciate your providing an easy way to build Risingwave with PGO, and experienced users will be able to do it on their own for their own usage scenarios. Another option is to optimize the Risingwave builds with a generic-enough profile. Providing PGO-optimized binaries could be a trickier task (since it requires preparing a good-enough profile) but as an option would be great to see too. Also, you could optimize distributed from the official sites binaries with PGO (if would be able to collect a generic profile) or even optimized cloud-based instances.

As an additional optimization, I suggest looking at LLVM BOLT. But from my experience, starting with PGO and then using BOLT would be better.

For the Rust projects, I recommend starting with cargo-pgo.

Describe alternatives you've considered

No response

Additional context

No response

fuyufjh commented 1 year ago

Thanks for your feedback!

We do have a plan of providing a "Performance Tuning Guide".

And thanks a lot for sharing the ClickHouse's PGO and cargo-pgo. We'll do some investigation.

github-actions[bot] commented 10 months ago

This issue has been open for 60 days with no activity. Could you please update the status? Feel free to continue discussion or close as not planned.