search

Shopify / yjit-metrics

"Tasks for benchmarking, building and collecting stats for YJIT"
MIT License
14 stars 9 forks source link

Add a button that just runs yjit-bench interface #222

Open rwstauner opened 9 months ago

rwstauner commented 9 months ago

The configurable_run_job uses yjit-metrics/basic_benchmark.rb (designed for collecting metrics to populate the website) rather than yjit-bench/run_benchmark.rb (which is useful for testing WIP branches).

It would be nice if there was a button (new jenkins job) to be able to run yjit-bench with configurable ruby builds so you can easily get timings from the benchmark server.

It will need to take config for building multiple rubies and then compile them like the existing job does. The data files should be easily accessible (uploaded as artifacts to the job).

We should try to get it into the same build queue as the other jobs so that we aren't running multiple benchmarks at the same time (which also means that you might schedule the job and have to wait 9 hours for it to finish).

noahgibbs commented 9 months ago

You've probably already found the Jenkinsfiles in continuous_reporting. But if you have any trouble with the setup, let me know and I can show you around it. I'm not sure why we're seeing cases where multiple benchmark runs can start at once, but I have seen it do it, so I'm not sure I trust Jenkins' infrastructure for that. You can also write a file, then watch for it in other jobs and block... But of course, it's possible for that to get jammed forever.

rwstauner commented 9 months ago

Good to know, thanks! I'll try to watch for that occurring.