search

scylladb / scylla-bench

42 stars 34 forks source link

Add possibility to stop execution after specific number of errors #110

Closed vponomaryov closed 1 year ago

vponomaryov commented 1 year ago

Now it is possible to specify number of errors after which a stress runner will stop the execution. To do it just add following cmd option to the s-b command: -error-limit n If it is not set it gets default value as 0 which means no limit (status quo).

vponomaryov commented 1 year ago

Tested here: https://jenkins.scylladb.com/view/staging/job/scylla-staging/job/valerii/job/longevity-large-partition-asymmetric-cluster-3h/39/consoleFull

vponomaryov commented 1 year ago

Latest state of this PR was tested here: https://jenkins.scylladb.com/view/staging/job/scylla-staging/job/valerii/job/longevity-large-partition-asymmetric-cluster-3h/40/consoleFull It works. Example of output of the failed loader:

1h53m16.9s    5589   55890      0 24ms   24ms   23ms   22ms   22ms   21ms   18ms   
1h53m17.9s    5597   55970      0 25ms   24ms   23ms   22ms   22ms   21ms   18ms   
1h53m18.9s    5591   55910      0 30ms   27ms   23ms   23ms   22ms   21ms   18ms   
2022/11/04 18:24:43 EOF
1h53m19.6s    5568   55680      0 30ms   29ms   24ms   22ms   22ms   21ms   18ms   
Results
Time (avg):  1h53m19.580764851s
Total ops:   33454696
Total rows:  334535935
Total errors:    1
Operations/s:    +Inf
Rows/s:      +Inf
raw latency :
  max:       50.593791ms 
  99.9th:    31.064063ms 
  99th:      23.789567ms 
  95th:      22.544383ms 
  90th:      22.151167ms 
  median:    20.971519ms 
  mean:      20.320071ms
c-o fixed latency :
  max:       50.593791ms 
  99.9th:    31.064063ms 
  99th:      23.789567ms 
  95th:      22.544383ms 
  90th:      22.151167ms 
  median:    20.971519ms 
  mean:      20.323227ms

Following critical errors were caught during the run:
    Error limit (maxErrors) of 1 errors is reached
    Error limit (maxErrorsAtRow) of 1 errors is reached