search

rust-lang / rustc-perf

Website for graphing performance of rustc
https://perf.rust-lang.org
616 stars 148 forks source link

Any way to use this repo for non-Rust project? #1566

Open hehaoqian opened 1 year ago

hehaoqian commented 1 year ago

I am in a team for compiler development.

Not related to Rust, but developing under MLIR framework

Finding a way to track the performance of our compiler (both compilation speed and quality)

Any suggestion how to make use of this repo for our purpose?

Kobzol commented 1 year ago

Hi, in theory, it could be possible, but I'm not sure if it's practical. Most of the code in this repository is specialized heavily for the Rust compiler. What exactly would you like to repurpose? The UI? Or the profiling/benchmarking code?

hehaoqian commented 1 year ago
  1. I like the UI, which provides the trending graph, and provide easy way to compare commits.
  2. We are current using the llvm-lnt framework, as show in http://lnt.llvm.org/, and its documentation It provides easy way to customize schema by yaml, so I can just simply submit json to its database, to submit a performance record. But I do not like its website, for its way to compare performance. llvm-lnt is using Jinja framework, its trending graph is hard to understand, and rendering is slow

Everyone in our compiler team is newbie to database and website frontend. So we just want to make use of open source solutions

I am considering to modify the "site" directory repository, to read the SQL database managed by llvm-lnt, any suggestion?

nnethercote commented 1 year ago

I haven't tried it, but https://github.com/bencherdev/bencher is a project that does generic benchmarking. It may be more suitable.

Kobzol commented 1 year ago

Indeed there are some general tools available that might be a better fit (I even developed one a few years ago, but it's not maintained anymore).

That being said, if you want to use the exact same UI as rustc-perf does, you might be able to rip out the necessary HTML/JS/CSS and implement your own backend for it. That shouldn't be too complicated. If you go that route, I would suggest to use a version of the repository a few months old, because recently we have switched to a Rust-based templating engine for the web, which might complicate things a bit for you. Before it was just raw static HTML pages.

hehaoqian commented 1 year ago

I have tried https://github.com/bencherdev/bencher, but it looks too new to be production ready.

We need the service to be self-hosted, for the security reason

ChatGpt suggests me for the following solutions:

Looks like they are much more powerful, but will take me a while to learn how to use it

hehaoqian commented 1 year ago

Also looked at https://grafana.com/ Our IT uses that to track the loading of the machines. It looks like it has powerful UI.