Improve usability of performance testing framework

ray6080 commented 10 months ago

Here are several issues we can improve for better readability of benchmark results.

[x] Show the cypher query directly for the website.
[x] Show PR number of commit message or link to the commit/pr so it is easier to trace back to what happened.
[x] Report database binary size.
[x] Summarized diff report in CI to show the performance difference compared to latest master if there are noticeable changes.

More functionalities:

[x] multiple datasets. we will add more datasets to benchmark runs, and currently we don't have a good way to show results based on different datasets as I understand. Will need to add a category for datasets too later.

ray6080 commented 5 months ago

[ ] Another good feature to add is generating flamegraph for specific queries. This can be triggered manually when needed. Right now, it is non-trivial to get the flamegraph manually on server, as it requires manual efforts to run the script and scp files around.

andyfengHKU commented 2 months ago

The basic ones are done. For flame graph we can add when we enter the next performance improvement stage. I'm closing this issue for now.

kuzudb / kuzu