aerospike / act

Aerospike Certification Tool
www.aerospike.com
Other
188 stars 50 forks source link

ACT TEST1 and TEST2 #38

Closed ssabnis closed 5 years ago

ssabnis commented 5 years ago

In the ACT TEST 1 and TEST 2 you never mentions or provide the number threads/queue. A sample will do. Or the number thread irrelevant for the ACT?

If I am using 56 Core, 256 GB, what would be recommended number of threads.

Also for certification do I need to run "storage" and "index" or just storage will do?

Thanks

Alb0t commented 5 years ago

The defaults are in the readme:


Fields that you will Rarely or Never Change:
num-queues Total number of transaction queues. Default is number of cores, detected by ACT at runtime.

threads-per-queue Number of threads per transaction queue. If a device is failing and there is a large discrepancy between transaction and device speeds from the ACT test you can try increasing the number of threads. Default is 4 threads/queue.```
ssabnis commented 5 years ago

Thanks much @Alb0t ,

Also for certification do I need to run "storage" and "index" or just storage will do?

Thanks again.

Alb0t commented 5 years ago

You probably just want storage.

If you are wanting to use all flash/index on disk, that's what that option is for. If you want to pursue that, you should get in touch with enterprise support.

On Sun, Jan 13, 2019, 9:36 PM ssabnis <notifications@github.com wrote:

Thanks much @Alb0t https://github.com/Alb0t ,

Also for certification do I need to run "storage" and "index" or just storage will do?

Thanks again.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/aerospike/act/issues/38#issuecomment-453900520, or mute the thread https://github.com/notifications/unsubscribe-auth/Am0CAr7jtpBW9EMKBIer2E-iOH912kbbks5vDAk4gaJpZM4Z9YQ6 .

ssabnis commented 5 years ago

Thanks @Alb0t , As I understand, you are saying index stored on Flash instead of memory. If that us true, I want to do that as well. What is that I need from enterprise support to test the performance for index. Is that something I can use ATC itself. Just wondering.

The storage I am using is high performance NVMe Flash drives.

Thanks again for help.

Alb0t commented 5 years ago

You can open a case with them. You'll need a license for all flash, and they can help advise on use case and performance testing. Do you have enterprise support?

ssabnis commented 5 years ago

Thanks @Alb0t ,

I do not have enterprise support. We are All Flash shared storage device company. We are not an application company. Please advise how/whom to contact for performance testing license.

Thanks

Alb0t commented 5 years ago

I think you misunderstand. You can use Aerospike free with flash storage. ACT testing is free for you use. The index on disk is a special feature that you can use with an enterprise license and only applies on certain scenarios. If you want to talk to them about enterprise features, just go here and fill out your details. They're real nice folks. https://www.aerospike.com/forms/contact-us/

ssabnis commented 5 years ago

Thanks @Alb0t , I understand now.

BTW; What is the difference between reads and device-reads sections.? Can you help. Thanks.

    reads                                       device-reads
    %>(ms)                                      %>(ms)

slice 1 8 64 512 4096 32768 1 8 64 512 4096 32768


1     0.03   0.00   0.00   0.00   0.00   0.00     0.02   0.00   0.00   0.00   0.00   0.00
2     0.02   0.00   0.00   0.00   0.00   0.00     0.01   0.00   0.00   0.00   0.00   0.00
ssabnis commented 5 years ago

@Alb0t .

I get the following error for reads setting 220000.

ERROR: large block writes can't keep up drive(s) can't keep up - test stopped

How do I interpret this.

Thanks

Alb0t commented 5 years ago

exactly what it says. your drive couldn't keep up.

ssabnis commented 5 years ago

@Alb0t ,

Thank you. Can you tell me how interpret the following, what is the difference

reads                                       device-reads
%>(ms)                                      %>(ms)

Thanks

Alb0t commented 5 years ago

I'd suggest moving this discussion to https://discuss.aerospike.com The readme on the front page goes over this a bit, so other members can chime in too. There isn't as much participation on github.

Have you gone over the README? Specifically, this section with the example.. The reads table shows read transaction latency (end to end) and the device-reads table at the right shows raw device (small) read latency.

ssabnis commented 5 years ago

Thanks @Alb0t ,

I missed that section, thanks for the help. I will move the discussion to other thread

gooding470 commented 5 years ago

@Albot, thanks so much for answering these questions.

BTW, we are aware of the issue where an error line in the log output that interrupts a histogram causes the latency analyzer to do weird things. At some point we will try to cover this case, since it may not be generally true that an error in the log means the test is meaningless. I just need to re-learn Python (again) and go back in there...

We also still intend to get to your script to automate testing...

Andy

ssabnis commented 5 years ago

Thanks @Alb0t @gooding470 ,

I am capping at 200k/100k r/w setup, after that system behaves. Here are numbers so far into the testing.

image

ssabnis commented 5 years ago

Hi @gooding470 ,

What is actually happening in the system for the below error.


ERROR: large block writes can't keep up
drive(s) can't keep up - test stopped
mtendjou commented 5 years ago

Would seem your disk is not keeping up and the write blocks are backing up (not able to be written down to storage fast enough), causing the test to stop.

ssabnis commented 5 years ago

Thanks @mtendjou

ssabnis commented 5 years ago

@gooding470, @mtendjou

What are the settings in the ACT configuration file to simulate the following test as shown for GCP.

Thanks,

image

Alb0t commented 5 years ago

@ssabnis are you asking how to create the graph? If you want to change the workload, just edit the config file..

ssabnis commented 5 years ago

@Alb0t ,

I am looking for config settings. like 100% read, 80% read and 20% write, config are not clear on this.

gooding470 commented 5 years ago

@ssabnis -- your graph only shows read-write ratios, not the actual loads. For example, you could get a read-write ratio of 50%-50% by configuring both read requests per second and write requests per second to 1,000. But you could also get the same 50-50 ratio by configuring them both to 100,000. That would likely make a significant difference in the latencies. Anyhow, to adjust your read-write loads and therefore ratios, just adjust the requests per second for each in the config file.