search

mulbc / gosbench

Distributed S3 benchmarking tool - Replacement of Cosbench
GNU General Public License v3.0
42 stars 16 forks source link

Performance drop when object size is large #19

Closed microyahoo closed 9 months ago

microyahoo commented 1 year ago

cherry-pick from https://github.com/mulbc/gosbench/pull/10/files

Signed-off-by: Liang Zheng zhengliang0901@gmail.com

microyahoo commented 1 year ago

@mulbc PTAL, thanks.

microyahoo commented 1 year ago

hi @mulbc, I noticed ceph team runs a simple comparison between COSBench and GOSBench.

From these charts, it’s apparent that the performance metrics for 64KB and 1MB objects are similar for both tools. For the 32MB objects, COSBench reports significantly higher performance metrics than GOSBench. One possible explanation for this disparity is that GOSBench is using multipart uploading and downloading. This feature is not in effect for objects smaller than 5MB and thus would only be visible with the 32MB objects. As of this writing, we’re still looking into this issue.

Is is feasible to replace uploader with PutObject to add object to bucket? Any comments are welcome, thanks.