Note that in the passed variant the add node operation after decommission was started at 11:39.
So, decommission under heavy write load took about 2h15m whereas current timeout is 1h20m.
Steps to Reproduce
Setup a DB cluster with tablets
Run heavy write load with large partitions
Run disrupt_decommission_streaming_err nemesis
Expected behavior: SCT waits proper amount of time
Actual behavior: SCT raises timeout error too early
Impact
False negative
How frequently does it reproduce?
100%
Installation details
SCT Version: master
Scylla version (or git commit hash): master/6.3
Issue description
Nemesis
disrupt_decommission_streaming_err
times out in the test with enabled tablets:Argus:
Looking at the cluster state everything was going ok all that time while timeout was not reached.
So, I increased the timeout for it and ran another test here: scylla-staging/valerii/vp-longevity-large-partition-200k-pks-4days-gce-test#4
After timeout increase the nemesis passed:
Note that in the passed variant the
add node operation after decommission
was started at11:39
. So, decommission under heavywrite load
took about2h15m
whereas current timeout is1h20m
.Steps to Reproduce
disrupt_decommission_streaming_err
nemesisExpected behavior: SCT waits proper amount of time
Actual behavior: SCT raises timeout error too early
Impact
False negative
How frequently does it reproduce?
100%
Installation details
SCT Version: master Scylla version (or git commit hash): master/6.3
Logs