kv: hand off a number of roachtests

cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.

https://www.cockroachlabs.com

Other

30.07k stars 3.8k forks source link

kv: hand off a number of roachtests #82060

Closed tbg closed 2 years ago

tbg commented 2 years ago

At the time of writing, KV owns many roachtests:

bin/roachtest list tag:owner-kv | grep -cF '[kv]'
220

We expect that for a lot of them, we can transfer ownership at some point in the near future.

I prepared a spreadsheet with a v0 of proposed movement: https://docs.google.com/spreadsheets/d/1w_zx8W1ObHVL1oMXNjh4LTcEMkTHyCycbGSnji1BvCk/edit#gid=2080879335

cc @lidorcarmel (for visibility) @srosenberg (since TE is the would-be owner for a lot of the tests and we should discuss the specifics of such a handover). Note that PERF is just TE today, but I thought it would make sense to make that distinction early.

Jira issue: CRDB-16193

nvanbenschoten commented 2 years ago

@tbg what are the next steps here? To determine which tests can be handed off? To relabel those tests?

tbg commented 2 years ago

Test-Eng will take the perf tests mid-July and we'll hand of another batch after that (details TBD), possibly the version-upgrade tests.

mwang1026 commented 2 years ago

Removing KV then since seems like Test Eng will take the lead on actually executing the change. Let me know if this is wrong @srosenberg

tbg commented 2 years ago

I was going to make the PR.

blathers-crl[bot] commented 2 years ago

cc @cockroachdb/replication

tbg commented 2 years ago

Still need to talk the schedule and selection through with @srosenberg but this is the rest that I believe should be owned by test-eng:

[x] Jepsen - if they fail then mostly due to flakes (apt-get etc). For actual test failure tutorial can reach out to KV if it fails (the average KV person doesn’t know how to investigate these either, but Tobi or Nathan do)
[x] tpcc/headroom/n4cpu16
[x] tpcc/mixed-headroom/n5cpu16
[x] tpcc/w=100/nodes=3/chaos=true
[x] version/mixed/nodes=3
[x] version/mixed/nodes=5
[x] weekly/tpcc/headroom

tbg commented 2 years ago

Here's what's left after #86300

$ roachtest list | grep -F '[kv]' | awk '{print $1}' | grep -Eo '^[^/]+' | uniq -c | sort
   1 autoupgrade
   1 drain-and-decommission
   1 drop
   1 gossip
   1 hotspotsplits
   1 inconsistency
   1 indexes
   1 ledger
   1 network
   1 overload
   1 queue
   1 restart
   1 slow-drain
   2 copy
   2 copyfrom
   2 kv50
   2 replicagc-changed-peers
   2 roachmart
   3 replicate
   3 transfer-leases
   4 loqrecovery
   4 splits
   5 decommission
   5 rebalance
   6 clock
   6 kv
   8 kv0bench
  10 decommissionBench
  11 acceptance
  13 follower-reads