search

lmnr-ai / lmnr

Laminar - open-source all-in-one platform for engineering AI products. Traces, Evals, Datasets, Labels. YC S24.
https://www.lmnr.ai
Apache License 2.0
959 stars 46 forks source link

Evaluate using more optimizations: LTO, PGO, PLO #10

Open zamazan4ik opened 2 months ago

zamazan4ik commented 2 months ago

Hi!

Found your project several days ago - nice work! I have several suggestions about how the project can be improved from the performance perspective. How critical performance questions at the current LMNR project lifecycle are ofc up to you ;)

I noticed that for Rust parts Link-Time Optimization (LTO) for the project is not enabled. I suggest switching it on since it will reduce the binary size (always a good thing to have) and will likely improve the application's performance (a lot or not - it depends).

I suggest enabling LTO only for the Release builds so as not to sacrifice the developers' experience while working on the project since LTO consumes an additional amount of time to finish the compilation routine. If you think that a regular Release build should not be affected by such a change as well, then I suggest adding an additional release-lto (actual naming is completely up to you) profile where additionally to regular release optimizations LTO also will be added. Such a change simplifies life for maintainers and others interested in the project persons who want to build the most performant version of the application. Using ThinLTO also should help).

If you are ready to invest more resources into improving the project's overall performance, I can suggest taking a look at Profile-Guided Optimization (PGO) and Post-Link Optimization(PLO) - I write about them a lot in my repo: https://github.com/zamazan4ik/awesome-pgo (and this article). PGO and PLO are very promising optimizations in your case that can help to achieve 10-20% (or even more) performance wins to you. Since LMNR uses 3rd party projects like ClickHouse and PostgreSQL, I can suggest trying to optimize their images with LTO, PGO, PLO too since it can bring a better experience for the whole platform (SaaS case) and on-premise setups. For ClickHouse, PostgreSQL, and many-many other databases (and projects) PGO benchmarks are available at the awesome-pgo repo

Thank you.

dinmukhamedm commented 1 month ago

Hello! Great suggestions, for certain. Unfortunately, I cannot see our team prioritizing this immediately, but we are sure open for contributions in this direction. We are still working out our contribution guidelines, but optimizations are always welcome😄