Can not write to mounted pvc

Hello,

I just went throug the installation description. The S3 is a mino instance without TLS and runs outside the kubernetes cluster. It's a demo installation. I can run a minio/mc docker inside the kubernetes and can connect to minio, create and delete buckets. So far so good networking works.

I'm running the example. So I use the secret.yaml, the pvc.yaml and the pod.yaml

So publishing user/password here is not critical The secret.yaml looks like

apiVersion: v1
kind: Secret
metadata:
  # Namespace depends on the configuration in the storageclass.yaml
  namespace: kube-system
  name: csi-s3-secret
stringData:
  accessKeyID: minioadmin
  secretAccessKey: minioadmin
  # For AWS set it to "https://s3.<region>.amazonaws.com"
  endpoint: 'http://192.168.16.131:9000/'
  # If not on S3, set it to ""
  region: ''

The pvc.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: csi-s3-pvc
  namespace: default
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi
  storageClassName: csi-s3

The pvc is created fine. Calling kubectl describe pvc csi-s3-pvc does not show any errors and its bound, access mode rwo.

Now starting the nginx

apiVersion: v1
kind: Pod
metadata:
  name: csi-s3-test-nginx
  namespace: default
spec:
  containers:
   - name: csi-s3-test-nginx
     image: nginx
     volumeMounts:
       - mountPath: /var/lib/www/html
         name: webroot
  volumes:
   - name: webroot
     persistentVolumeClaim:
       claimName: csi-s3-pvc
       readOnly: false

The nginx starts fine, logs and describe shows no errors

Next opening a shell with

kubectl exec -it csi-s3-test-nginx -- bash
mount | grep fuse

fuse shows a mounted pvc on /var/lib/www/html. Using the UI of minio shows a fresh created bucket in read/write mode. So everything is fine until now. But comaring with the Readme. I get a different answer.

root@csi-s3-test-nginx:/# mount | grep fuse
:s3:pvc-ca79e362-0394-4fc5-9909-ba8e0ddc436a/csi-fs on /var/lib/www/html type fuse.rclone (rw,nosuid,nodev,relatime,user_id=0,group_id=0,allow_other)

In the readme the answer to the mount should start with

s3fs on /var/lib/www/html type log ....

Is there something that went wrong, because I got a different answer?

When executing

root@csi-s3-test-nginx:/#  ls /var/lib/www/html

I get an error

ls: reading directory '/var/lib/www/html': Input/output error

So what is going on there? I should be able read,write on that mounted pvc, but it does not do anything.

Is there something I have overseen? Or is this behaviour as expected.

Any help welcome

Ok, so I've got it. I just used the storageclass.yaml as is from the clone git repo. This storageclasse uses the mounter: rclone and not s3sf.

So changing it to mounter: s3fs did the trick. And things working as expeced

---
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: csi-s3
provisioner: ch.ctrox.csi.s3-driver
parameters:
  # specify which mounter to use
  # can be set to rclone, s3fs, goofys or s3backer
  mounter: s3fs
  # to use an existing bucket, specify it here:
  # bucket: some-existing-bucket
  csi.storage.k8s.io/provisioner-secret-name: csi-s3-secret
  csi.storage.k8s.io/provisioner-secret-namespace: kube-system
  csi.storage.k8s.io/controller-publish-secret-name: csi-s3-secret
  csi.storage.k8s.io/controller-publish-secret-namespace: kube-system
  csi.storage.k8s.io/node-stage-secret-name: csi-s3-secret
  csi.storage.k8s.io/node-stage-secret-namespace: kube-system
  csi.storage.k8s.io/node-publish-secret-name: csi-s3-secret
  csi.storage.k8s.io/node-publish-secret-namespace: kube-system

So classical user error.

Repeat the steps in the readme and things are working. fine

Remark: rclone with minio is described here. So this is a describtion using rclone client. What this option does when using as mounter keeps unclear to me.

ctrox / csi-s3

Can not write to mounted pvc #65