search

cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
29.84k stars 3.77k forks source link

roachtest: override (performance) workload flags #62303

Closed erikgrinaker closed 4 weeks ago

erikgrinaker commented 3 years ago

It's currently not possible to override individual workload flags when run via roachtest. For example, it would be nice to override --duration for the kv95 workloads, but roachtest uses a fixed workload --duration 10m.

Jira issue: CRDB-2810

github-actions[bot] commented 1 year ago

We have marked this issue as stale because it has been inactive for 18 months. If this issue is still relevant, removing the stale label or adding a comment will keep it active. Otherwise, we'll close it in 10 days to keep the issue queue tidy. Thank you for your contribution to CockroachDB!

blathers-crl[bot] commented 1 year ago

cc @cockroachdb/test-eng

csgourav commented 1 month ago

There seems to be already enviornment variable way of updating duration exists for the two workloads, kv and ycbs, though not add in all tests registered fro them.

PR - https://github.com/cockroachdb/cockroach/commit/b8986e311587ae4a437747132a24ad6a18c0b7eb

Originally added to pass on duration for the kv and YCSB workload ROACHTEST_KV_FLAGS and ROACHTEST_YCSB_FLAGS to pass additional flags to the workloads in order to override the duration (e.g. ROACHTEST_KV_FLAGS="--duration 5m")

The other way would be using the roachtestflag, which would be global and would need making changes in all the tests. Global flag might be confusing for non workload type tests.

srosenberg commented 1 month ago

There seems to be already enviornment variable way of updating duration exists for the two workloads, kv and ycbs, though not add in all tests registered fro them.

Nice, I wasn't aware of that change. We still need to propagate these into the roachtest environment.

The other way would be using the roachtestflag, which would be global and would need making changes in all the tests. Global flag might be confusing for non workload type tests.

Agreed, let's leverage the env. vars. You'll need a way to pass them from the client (i.e., entity executing roachtest binary) into the (remote) environment of the workload binary. Note that a particular workload could either be run from the cockroach binary or the workload binary, as is the case with kv and ycsb; workload binary is a superset, i.e., it packages all workloads.

For the cockroach binary, env. vars. are injected via the startup script (start.sh); see ClusterSettings.Env and generateStartCmd. We'll probably have to do something similar for the workload binary since today it's not invoked via a wrapper script.

csgourav commented 1 month ago

I think we won't need to pass these two workload or cockroach binary. That is taken care in this change, while calling the workload binary itself, we are passing updated duration value fetched from the environment value.

Done in this PR https://github.com/cockroachdb/cockroach/commit/b8986e311587ae4a437747132a24ad6a18c0b7eb

To make use of this in the bisect_runner.sh script, we will have to modify that script to make use of the environment variable. Which is in this repo https://github.com/smg260/roachperf-bisect/tree/72e4ba7a29c2c9267addfdce6170cf6e8f282eda