seaweedfs / seaweedfs-csi-driver

SeaweedFS CSI Driver https://github.com/seaweedfs/seaweedfs
Apache License 2.0
210 stars 50 forks source link

SeaweedFS CSI Driver is really slow #174

Closed Gradlon closed 1 month ago

Gradlon commented 1 month ago

I did a bit of testing and found that the driver is really slow. In my setup I get about MB/s using dd if=/dev/zero of=/data/testfile bs=10M count=1 Normal mount gets around 82MB/S

services:
  volume:
    image: chrislusf/seaweedfs:3.69_large_disk_full
    container_name: seaweedfs-volume
    environment:
        TZ: Europe/Zurich
    network_mode: "host"
    command: >
      -v=2
      volume 
      -dataCenter=${DATACENTER} 
      -rack=${RACK} 
      -mserver=10.0.0.15:9333,10.0.0.11:9333,10.0.0.13:9333
      -ip=${IP} 
      -ip.bind=0.0.0.0 
      -port=9090  
      -metricsPort=9325
      -dir=/data/volumes/01,/data/volumes/02
      -max=0,0
      -idleTimeout=60
      -index=leveldbMedium
      -hasSlowRead=true
      -readBufferSizeMB=64
      -concurrentDownloadLimitMB=10000
      –concurrentUploadLimitMB=10000
      -fileSizeLimitMB=4096
    volumes:
      - /data/bin/curl:/bin/curl:z,ro
      - /data/seaweedfs/volumes:/data/volumes:Z
      - ./config:/etc/seaweedfs:Z,ro
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536 # maximum number of open files for the OpenSearch user, set to at least 65536 on modern systems
        hard: 65536
    healthcheck:
      test: [ "CMD", "curl", "--fail", "-I", "http://localhost:9090/healthz" ]
      interval: 3s
      start_period: 15s
      timeout: 30s
  ```
  services:

filer: image: chrislusf/seaweedfs:3.69_large_disk_full container_name: seaweedfs-filer network_mode: "host" command: '-v=2 filer -master=10.0.0.15:9333,10.0.0.11:9333,10.0.0.13:9333 -ip.bind=0.0.0.0 -metricsPort=9326 -collection=storage' volumes:

Docker version 27.0.3, build 7d4bcd8

there are two Storage servers and 3 Masters

Gradlon commented 1 month ago

Found the Issue. Make sure to set the Datacenter in the Plugin like in the Seaweed volume server.

docker plugin set seaweedfs-csi:swarm DATACENTER=XXX
      -dataCenter=XXX