opencurve / curve

Curve is a sandbox project hosted by the CNCF Foundation. It's cloud-native, high-performance, and easy to operate. Curve is an open-source distributed storage system for block and shared file storage.
https://opencurve.io
Apache License 2.0
2.33k stars 522 forks source link

curve-fuse's memory usage is unlimited when enabling `fakes3` #1593

Closed wu-hanqing closed 1 year ago

wu-hanqing commented 2 years ago

Describe the bug (描述bug)

the process runs for 30 minutes, and the memory usage is 17.7g

image

To Reproduce (复现方法)

client yaml

s3.ak: curve
s3.sk: curveadmin
s3.endpoint: xxx
s3.bucket_name: vdbench
container_image: xxx
mdsOpt.rpcRetryOpt.addrs: xxx
log_dir: /data/curve/d261fe4e-ee1f-11ec-b701-6c92bf5e4d94/logs
diskCache.diskCacheType: 2
data_dir: /data/curve/d261fe4e-ee1f-11ec-b701-6c92bf5e4d94/cache
fuseClient.cto: false
fuseClient.enableMultiMountPointRename: false
s3.fakeS3: true
core_dir: /data/curve/d261fe4e-ee1f-11ec-b701-6c92bf5e4d94/core

vdbench profile

fsd=fsd1,anchor=${MOUNTPOINT}/d1,depth=1,width=10,files=50,sizes=(100m,0),shared=yes,openflags=o_direct
fsd=fsd2,anchor=${MOUNTPOINT}/d2,depth=1,width=10,files=50,sizes=(100m,0),shared=yes,openflags=o_direct
fsd=fsd3,anchor=${MOUNTPOINT}/d3,depth=1,width=10,files=50,sizes=(100m,0),shared=yes,openflags=o_direct
fsd=fsd4,anchor=${MOUNTPOINT}/d4,depth=1,width=10,files=50,sizes=(100m,0),shared=yes,openflags=o_direct
fsd=fsd5,anchor=${MOUNTPOINT}/d5,depth=1,width=10,files=50,sizes=(100m,0),shared=yes,openflags=o_direct
fwd=fwd1,fsd=fsd*,threads=10,xfersize=(512,20,4k,20,64k,20,512k,20,1024k,20),fileio=random,fileselect=random,rdpct=50
rd=rd1,fwd=fwd*,fwdrate=max,format=restart,elapsed=99999999999999999,interval=1

Expected behavior (期望行为)

Versions (各种版本) OS: Compiler: branch: release2.3 commit id: 12ae8abb

Additional context/screenshots (更多上下文/截图)

before this, I also built curve-fuse from the master, and also faced the same problem

image

curve-fuse used up the memory and got killed.

ilixiaocui commented 2 years ago

image

From the results of perf, most of the memory is consumed in S3InfoMap. Using fake backend to write is very fast, much faster than the backend Inode brush speed, resulting in memory accumulation.