search

nodejs / benchmarking

Node.js Benchmarking Working Group
https://benchmarking.nodejs.org
161 stars 47 forks source link

select-compiler in benchmark script #301

Closed rvagg closed 4 years ago

rvagg commented 4 years ago

microbenchmark runs are now failing since the V8 upgrade, see https://github.com/nodejs/node/pull/30547#issuecomment-555635270, e.g. https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/458/

It needs select-compiler.sh to be run after checking out nodejs/node (since it checks Node version) so it can't be done in Jenkins and needs to be done here.

This file: https://github.com/nodejs/benchmarking/blob/master/experimental/benchmarks/community-benchmark/run.sh

Needs to have this somewhere after nodejs/node clone:

# select the appropriate compiler
curl -sLO https://raw.githubusercontent.com/nodejs/build/master/jenkins/scripts/select-compiler.sh
. ./select-compiler.sh
mhdawson commented 4 years ago

The benchmark job turns out to be a bit of a snowflake. Took me a while to figure out why the right compiler was not being selected.

It does not use a configuration matrix. For x64 select compiler uses the $nodes value which ends up not being set if you don't have a configuration matrix.

Going to manually set $nodes in the job for now.

We should probably refactor the job to make it more consistent with the other community jobs, but I don't have the cycles to do that now.

mhdawson commented 4 years ago

@rvagg I think I've fixed it. The PR's you'd tried to run it on before did not run but it looked like an issue applying the PR. Doing a run against master on a current PR seemed ok.