search

brianfrankcooper / YCSB

Yahoo! Cloud Serving Benchmark
Apache License 2.0
4.94k stars 2.24k forks source link

partition workload #589

Open bingzhang opened 8 years ago

bingzhang commented 8 years ago

From this link, https://github.com/brianfrankcooper/YCSB/wiki/Running-a-Workload-in-Parallel I know we can partition recorders in load phase. However, may I ask how to partition the workload in execution phase? For example, we have 20 recorders, operations from client1 will be in the range [0, 10], while client2 will be in [11, 20]. there will be no duplicate requests between multiple clients.

Thanks.

kruthar commented 8 years ago

Hi @bingzhang - The functionality you describe, partitioning the keysequence used for the transaction phase (run), is not possible at this time. I took a look at what it might take to implement it, and it certainly seems possible.

Would you be interested in making this contribution?

bingzhang commented 8 years ago

May I know which files I need to modify, if we want to control the number of duplicated requests between clients?

Thanks.

cmatser commented 8 years ago

Take a look at YCSB/core/src/main/java/com/yahoo/ycsb/workloads/CoreWorkload.java

I believe you could update the keychooser variable to create the IntegerGenerator to start a specified value instead of zero (which it does currently). It may make sense to reuse the insertstart property.

I'm curious about the use case for this scenario you are testing. Most data access isn't usually partitioned like this. My work usually focuses on how the db handles contention and consistency when multiple users are accessing and/or updating the same record. Would you mind telling a little about your partitioned scenario?

busbey commented 8 years ago

@bingzhang are you still interested in discussing your use case?

michaelrun commented 5 months ago

Is this requirement implemented now? I'm facing this issue, testing MongoDB shard cluster, running one router mongos, mongod CPU utilization is low, around 50%，add another router(mongos), CPU utilization goes higher, but to test 0.6 read, 0.3 update, 0.1 insert, failed to run two YCSB client due to partition issue. I'm using YCSB 0.17.