search

barry-boy / barry-boy.github.io

5 stars 1 forks source link

Juicefs fio 基准测试 #64

Open barry-boy opened 1 year ago

barry-boy commented 1 year ago

juicefs :

Test IOPS BW
random-read-4k 15961.0013 63844
random-write-4k 164.841218 659
read-4M 965.036283 3952788
write-4M 231.363924 947666

juicefs:

root@juicefs-test02-798f685bd4-5p872:/# dd if=/dev/zero of=/a bs=100M count=10
1048576000 bytes (1.0 GB, 1000 MiB) copied, 0.743292 s, 1.4 GB/s

root@juicefs-test02-798f685bd4-5p872:/# dd if=/dev/zero of=/a bs=1000M count=10
10485760000 bytes (10 GB, 9.8 GiB) copied, 65.2644 s, 161 MB/s
root@juicefs-test02-798f685bd4-5p872:/juicefs# dd if=/dev/zero of=./a bs=100M count=10
1048576000 bytes (1.0 GB, 1000 MiB) copied, 1.52067 s, 690 MB/s

root@juicefs-test02-798f685bd4-5p872:/juicefs# dd if=/dev/zero of=./a bs=1000M count=10
10485760000 bytes (10 GB, 9.8 GiB) copied, 14.1189 s, 743 MB/s

root@juicefs-test02-798f685bd4-5p872:/juicefs# rsync -avP ./b /
sending incremental file list
b
 10,485,760,000 100%  601.47MB/s    0:00:16 (xfr#1, to-chk=0/1)

sent 10,488,320,092 bytes  received 35 bytes  599,332,578.69 bytes/sec
total size is 10,485,760,000  speedup is 1.00

cephfs:

root@cephfs-7c6d8c8bfc-2t2ww:/cephfs# dd if=/dev/zero of=./b bs=100M count=10
1048576000 bytes (1.0 GB, 1000 MiB) copied, 0.517849 s, 2.0 GB/s
root@cephfs-7c6d8c8bfc-2t2ww:/cephfs# dd if=/dev/zero of=./b bs=1000M count=10
10485760000 bytes (10 GB, 9.8 GiB) copied, 4.86261 s, 2.2 GB/s

root@cephfs-7c6d8c8bfc-2t2ww:/cephfs# rsync -avP ./b /
sending incremental file list
b
 10,485,760,000 100%  645.16MB/s    0:00:15 (xfr#1, to-chk=0/1)

sent 10,488,320,091 bytes  received 35 bytes  635,655,765.21 bytes/sec
total size is 10,485,760,000  speedup is 1.00

rbd:

root@rbd-96b96f846-pr927:/rbd# dd if=/dev/zero of=./b bs=100M count=10
1048576000 bytes (1.0 GB, 1000 MiB) copied, 0.740311 s, 1.4 GB/s
root@rbd-96b96f846-pr927:/rbd# dd if=/dev/zero of=./b bs=1000M count=10
10485760000 bytes (10 GB, 9.8 GiB) copied, 12.1419 s, 864 MB/s

root@rbd-96b96f846-pr927:~# rsync -avP  /rbd/b1/ /data
sending incremental file list
./
b
 10,485,760,000 100%  813.60MB/s    0:00:12 (xfr#1, to-chk=0/2)

sent 10,488,320,115 bytes  received 38 bytes  839,065,612.24 bytes/sec
total size is 10,485,760,000  speedup is 1.00

参考文章:

barry-boy commented 1 year ago
juicefs测试： Test IOPS BW
random-read-4k 10921.502617 43686
random-write-4k 187.465354 749
read-4M 492.189378 2016007
write-4M 223.963532 917354

rbd 测试：

Test IOPS BW
random-read-4k 68087.891211 272351
random-write-4k 5278.986836 21115
read-4M 529.719104 2169729
write-4M 449.536854 1841302

cephfs 测试：

Test IOPS BW
random-read-4k 67437.637491 269750
random-write-4k 5403.166139 21612
read-4M 739.083363 3027285
write-4M 519.13815 2126389
barry-boy commented 1 year ago

juicefs

root@juicefs-test02-798f685bd4-5p872:/juicefs# juicefs bench ./ -p 4

+------------------+-----------------+--------------+
|       ITEM       |      VALUE      |     COST     |
+------------------+-----------------+--------------+
|   Write big file |    966.22 MiB/s |  4.24 s/file |
|    Read big file |   1757.63 MiB/s |  2.33 s/file |
| Write small file |   634.7 files/s | 6.30 ms/file |
|  Read small file |  5366.2 files/s | 0.75 ms/file |
|        Stat file | 15218.1 files/s | 0.26 ms/file |
+------------------+-----------------+--------------+

cephfs :

+------------------+------------------+--------------+
|       ITEM       |       VALUE      |     COST     |
+------------------+------------------+--------------+
|   Write big file |   10140.41 MiB/s |  0.40 s/file |
|    Read big file |   23082.57 MiB/s |  0.18 s/file |
| Write small file |   5983.9 files/s | 0.67 ms/file |
|  Read small file |  44394.6 files/s | 0.09 ms/file |
|        Stat file | 234011.5 files/s | 0.02 ms/file |
+------------------+------------------+--------------+

rbd:

Benchmark finished!
BlockSize: 1 MiB, BigFileSize: 1024 MiB, SmallFileSize: 128 KiB, SmallFileCount: 100, NumThreads: 4
+------------------+------------------+--------------+
|       ITEM       |       VALUE      |     COST     |
+------------------+------------------+--------------+
|   Write big file |    5793.88 MiB/s |  0.71 s/file |
|    Read big file |   22376.03 MiB/s |  0.18 s/file |
| Write small file |   6920.4 files/s | 0.58 ms/file |
|  Read small file |  41343.2 files/s | 0.10 ms/file |
|        Stat file | 271353.3 files/s | 0.01 ms/file |
+------------------+------------------+--------------+
barry-boy commented 1 year ago
Test IOPS BW
random-read-4k 34787.373754 139149
random-write-4k 35.636739 142
read-4M 972.090374 3981682
write-4M 219.572531 899369
Test IOPS BW
random-read-4k 4630.71231 18522
random-write-4k 38.593851 154
read-4M 855.972582 3506063
write-4M 260.377861 1066507
barry-boy commented 1 year ago

优化建议

image

barry-boy commented 11 months ago

不使用缓存盘

root@jfs-app-647458b74d-m8lvr:/juicefs# dd if=/dev/zero of=./b bs=100M count=10
10+0 records in
10+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 15.4683 s, 67.8 MB/s

使用缓存盘

/juicefs # dd if=/dev/zero of=./b bs=100M count=10
10+0 records in
10+0 records out
1048576000 bytes (1000.0MB) copied, 1.997702 seconds, 500.6MB/s
ceph rgw Test IOPS BW
random-read-4k 9796.580228 39186
random-write-4k 37.437964 149
read-4M 402.705679 1649482
write-4M 126.607155 518582