ydb-platform / nbs

Network Block Store
Apache License 2.0
52 stars 21 forks source link

[Filestore] Support local fs mode in filestore-vhost #1668

Open qkrorlqr opened 1 month ago

qkrorlqr commented 1 month ago

Sometimes we need to be able to pass local filesystems to the VMs - just like virtiofsd does. We want to have all the telemetry of filestore-vhost and we also want to use libaio, so vanilla virtiofsd doesn't suit our needs.

qkrorlqr commented 1 month ago

Prepare run env:

$ mkdir localfs && cd localfs && mkdir cfg && mkdir endpoints && mkdir sockets && mkdir data

Create config like this one:

$ cat cfg/vhost.txt 
VhostServiceConfig {
  ServiceEndpoints {
  }
  EndpointStorageType: ENDPOINT_STORAGE_FILE
  EndpointStorageDir: "endpoints"
  LocalServiceConfig {
    RootPath: "./data/"
  }
}
$ ./filestore-vhost --app-config ./cfg/vhost.txt --server-port 25001 --service local --mon-port 25011 --profile-file profile.log --local-service-port 25021

Create FS & start FS endpoint:

$ filestore-client create --filesystem localfs0 --cloud test_cloud --folder test_folder --block-size 4096 --blocks-count 107374182400 --server-address localhost --server-port 25021
$ filestore-client startendpoint --filesystem localfs0 --socket-path `pwd`/sockets/localfs0_0.sock --mount-seqno 0 --server-address localhost --server-port 25001

Run VM:

$ /usr/bin/qemu-system-x86_64 -L /usr/share/qemu -snapshot -cpu host -smp 16,sockets=1,cores=16,threads=1 -enable-kvm -m 16G -name debug-threads=on -qmp tcp:localhost:24001,server,nowait -object memory-backend-memfd,id=mem,size=16G,share=on -numa node,memdev=mem -netdev user,id=netdev0,hostfwd=tcp::24011-:22 -device virtio-net-pci,netdev=netdev0,id=net0 -object iothread,id=iot0 -drive format=qcow2,file=/tmp/test_rootfs.img,id=lbs0,if=none,aio=native,cache=none,discard=unmap -device virtio-blk-pci,scsi=off,drive=lbs0,id=virtio-disk0,iothread=iot0,bootindex=1 -nographic -gdb tcp::24021 -chardev socket,path=/root/localfs/sockets/localfs0_0.sock,id=vhost0,reconnect=1 -device vhost-user-fs-pci,chardev=vhost0,id=vhost-user-fs0,tag=fs0
qkrorlqr commented 1 month ago

TODO: