scylladb / scylla-bench

43 stars 36 forks source link

Add a new workload or parameter for range-deletions #94

Open yarongilor opened 2 years ago

yarongilor commented 2 years ago

The current available workload and parameters of scylla-bench doesn't support specifying operation of delete-range.

This is very important for testing of range-Thobstones revised mechanizm in a real-life scenario, using SCT infra-structure.

fruch commented 1 week ago

delete-range is much more complicated the rest of the use case s-b is doing, and needs to be in complete sync of what the test is doing.

for any implementation on s-b end, we'll need much more concrete requirements of what is the logic expected for this "real-life" scenario

fruch commented 1 week ago

today we can recommend using latte to implement a very specific user defined case like deletions

yarongilor commented 1 week ago

today we can recommend using latte to implement a very specific user defined case like deletions

@fruch , is there any documentation reference for writing any such use case with latte?

What's the repo for latte support? sct? should this issue moved to other repo?

Another example for this need is in test of https://github.com/scylladb/scylla-cluster-tests/pull/8948 - it currently write s-b large partitions, then delete it "manually" one by one, what takes an hour.

fruch commented 1 week ago

today we can recommend using latte to implement a very specific user defined case like deletions

@fruch , is there any documentation reference for writing any such use case with latte?

What's the repo for latte support? sct? should this issue moved to other repo?

this is the repo for latte: https://github.com/scylladb/latte

regardless it's not a issue for latte, it's for you to write the script you need you can find several examples in: https://github.com/scylladb/scylla-qa-internal/tree/master/custom_d1

Another example for this need is in test of scylladb/scylla-cluster-tests#8948 - it currently write s-b large partitions, then delete it "manually" one by one, what takes an hour.

again I don't know what is the exact requirement you have there, but you probably can script it in rune

vponomaryov commented 1 week ago

today we can recommend using latte to implement a very specific user defined case like deletions

@fruch , is there any documentation reference for writing any such use case with latte? What's the repo for latte support? sct? should this issue moved to other repo?

this is the repo for latte: https://github.com/scylladb/latte

regardless it's not a issue for latte, it's for you to write the script you need you can find several examples in: https://github.com/scylladb/scylla-qa-internal/tree/master/custom_d1

Another example for this need is in test of scylladb/scylla-cluster-tests#8948 - it currently write s-b large partitions, then delete it "manually" one by one, what takes an hour.

again I don't know what is the exact requirement you have there, but you probably can script it in rune

Example of the latte stress commands currently used in SCT:

It utilizes the following rune script:

Then, I looked at the PR which requires this feature here: https://github.com/scylladb/scylla-cluster-tests/pull/8948 The use case can easily be covered with latte. Existing custom-d1/workload2 test case we have in SCT is more complex than it is needed in your case. So, it should be fairly easy to do your rune script just by using the existing mentioned one as an example.

@fruch

delete-range is much more complicated the rest of the use case s-b is doing, and needs to be in complete sync of what the test is doing.

Even being agree that latte is more powerful, I disagree on the statement that delete-range is much more complicated for S-B. Delete operations don't fail if data, it references, are absent. So, it means we can simply add delete and delete_batch modes similar to write and write-batch ones we already have in scylla-bench. Probably we will need to add 1 additional option saying how exactly we delete data - by rows or by partitions.