minio / warp

S3 benchmarking tool
GNU Affero General Public License v3.0
553 stars 102 forks source link

can surrport multi bucket? #299

Closed lygn128 closed 8 months ago

lygn128 commented 8 months ago

can surrport multi bucket? use hash to put data to multi bucket?

harshavardhana commented 8 months ago

what difference will that make?

lygn128 commented 8 months ago

when i use one bucket the cluster qps max 4000 while disk and cpu not high, so i want to try multle buckets can reach a high qps? the storage cluster is minio, thank you very much.

harshavardhana commented 8 months ago

There is no such that two buckets will give you higher QPS, each bucket uses same hardware there is nothing more you will gain by WRITING to two buckets v/s same bucket.

lygn128 commented 8 months ago

There is no such that two buckets will give you higher QPS, each bucket uses same hardware there is nothing more you will gain by WRITING to two buckets v/s same bucket.

thank you very much. i have another question: when i use minio, can large directories(HDD disk, one directory have 1500000+ sub files or directories ) cause weak write or read performance? thank you very much.

harshavardhana commented 8 months ago

HDDs cannot be use for IOPs intensive tasks. You need to move to NVMe for better performance.

lygn128 commented 8 months ago

HDDs cannot be use for IOPs intensive tasks. You need to move to NVMe for better performance.

thank you very much, i have a cluster which 600 hdd drivers(6 servers), but when put 64K size file benchark, only get a 4000 qps bench result, cpu and memory and io is not high, do you think 4000 qps is a reasonable result? thank you very much!

klauspost commented 8 months ago

@lygn128 You can reach out on hello@min.io and we can help you assist with your benchmarking/setup.