With the addition of https://github.com/cockroachdb/cockroach/pull/104300, roachprod-microbench will be able to export microbenchmark results to open metrics format. The output is written to stdout, using the export sub-command.
Thus the weekly microbenchmark script [1] will need to be updated to direct the output to a file.
This file needs to be placed in a gcs bucket [2] which is monitored by another process.
This process will take the uploaded file and load it into bigtable via promtable.
The file is moved to a processed folder within the bucket, so we should ensure the naming is unique. Since these files are small it would be good to keep a history for a while at least.
The command can take labels as well which it will pass along in the metrics (this is useful to categorise the metrics for later use when querying). It also takes a timestamp. The implementer should supply both of these. The timestamp from the git revision and branch the microbenchmarks ran for might be a good consideration (others could be cloud & machine type). The labels could include on what machine type the microbenchmarks ran and other related info (possibly the branch etc.).
Also Note: We might not want this to happen if someone is running the job ad-hoc for their own purposes. There might be logic required to prevent sending the results for ad-hoc (non weekly) runs.
With the addition of https://github.com/cockroachdb/cockroach/pull/104300,
roachprod-microbench
will be able to export microbenchmark results to open metrics format. The output is written tostdout
, using theexport
sub-command.Thus the weekly microbenchmark script [1] will need to be updated to direct the output to a file. This file needs to be placed in a
gcs bucket
[2] which is monitored by another process. This process will take the uploaded file and load it intobigtable
viapromtable
.The file is moved to a processed folder within the bucket, so we should ensure the naming is unique. Since these files are small it would be good to keep a history for a while at least.
The command can take labels as well which it will pass along in the metrics (this is useful to categorise the metrics for later use when querying). It also takes a timestamp. The implementer should supply both of these. The timestamp from the git revision and branch the microbenchmarks ran for might be a good consideration (others could be cloud & machine type). The labels could include on what machine type the microbenchmarks ran and other related info (possibly the branch etc.).
Also Note: We might not want this to happen if someone is running the job ad-hoc for their own purposes. There might be logic required to prevent sending the results for ad-hoc (non weekly) runs.
[1] https://github.com/cockroachdb/cockroach/blob/master/build/teamcity/cockroach/nightlies/microbenchmark_weekly.sh [2] gs://cockroach-testeng-metrics/omloader/incoming
Jira issue: CRDB-28690