The bench bot was originally created to specifically be used for the node-bench utility. I have, over the last month or so, hacked on top of it the ability to also be used for the runtime benchmarks process.
But these hacks have mostly been... hacky, and some better design and architecture could be used to make the whole tool more robust, easy to upgrade, and more usable.
I suggest the following specification for how users should interact with the bot:
/benchmark (interchangeable with `/bench`)
or
/benchmark help
Return a list of available commands as a Github comment.
/benchmark status
Return availability status and other information about the running processes and queue.
/benchmark specs
Return the machine specifications of the benchmarking device.
/benchmark node <benchmark_filters>
Run a node benchmark using the benchmark filters.
Only whitelisted users should be able to trigger this command.
/benchmark runtime <runtime> <pallet>
Run a runtime benchmark for a specific runtime using the default flags.
The bench bot was originally created to specifically be used for the
node-benchutility. I have, over the last month or so, hacked on top of it the ability to also be used for the runtime benchmarks process.But these hacks have mostly been... hacky, and some better design and architecture could be used to make the whole tool more robust, easy to upgrade, and more usable.
I suggest the following specification for how users should interact with the bot:
Return a list of available commands as a Github comment.
Return availability status and other information about the running processes and queue.
Return the machine specifications of the benchmarking device.
Run a node benchmark using the benchmark filters.
Only whitelisted users should be able to trigger this command.
Run a runtime benchmark for a specific runtime using the default flags.
runtimeoptions: Substrate, Polkadot, Kusama, Westend, Template(?)palletoptions: Either a specific pallet name orall, which should do some magic to query the available benchmarks on the machine and run them all.Only whitelisted users should be able to trigger this command.
Run a benchmark with a specific set of flags (thus no need to also specify the runtime).
Only whitelisted users should be able to trigger this command.
Cancel a running benchmark.
Only if you are the person who initiated the benchmark or you are on some whitelist (like Parity Tech Core Devs)
Clear the entire queue. Only whitelisted users should be able to trigger this command.
Ideally, update the running process using the latest on the master branch of this repo and restart the bot.