search

yandex-cloud / k8s-csi-s3

GeeseFS-based CSI for mounting S3 buckets as PersistentVolumes
Other
581 stars 99 forks source link

Regarding the problem of repeated creation and hanging of pvc #118

Open lgy1027 opened 6 months ago

lgy1027 commented 6 months ago

Hello, I have a problem now that when multiple pods mount the same pvc, an error will be reported. The error message is as follows

d/system/geesefs-b20240529175837141bueysv.service.d: file exists
I0530 05:50:07.413362       1 utils.go:97] GRPC call: /csi.v1.Node/NodeGetCapabilities
I0530 05:50:07.417568       1 utils.go:97] GRPC call: /csi.v1.Node/NodeGetCapabilities
I0530 05:50:07.418593       1 utils.go:97] GRPC call: /csi.v1.Node/NodeGetCapabilities
I0530 05:50:07.419488       1 utils.go:97] GRPC call: /csi.v1.Node/NodeStageVolume
I0530 05:50:07.425608       1 geesefs.go:150] Starting geesefs using systemd: /var/lib/kubelet/plugins/ru.yandex.s3.csi/geesefs -f -o allow_other --endpoint http://10.0.101.61:32000 --setuid 65534 --setgid 65534 b20240529175837141bueysv: /var/lib/kubelet/plugins/kubernetes.io/csi/ru.yandex.s3.csi/3725d13a77a36f58d2aa92239796a86c72ec85d37d5370f69f3e7ed907602550/globalmount
E0530 05:50:07.428679       1 utils.go:101] GRPC error: Error starting systemd unit geesefs-b20240529175837141bueysv.service on host: mkdir /run/systemd/system/geesefs-b20240529175837141bueysv.service.d: file exists

It will be no problem if you delete the existing directory, but you will still have this problem when multiple pods mount a pvc.

Noah-Huppert commented 5 months ago

I had a similar error (ie., file exists) where a PVC would fail to attach to a pod and cause the pod to not create. I ended up having to delete the PVC, PV, and completely delete and reinstall k8s-csi-s3 for it to work again. I would love any hints about what this file exists error is caused by so I could debug further.

vitalif commented 5 months ago

Hi, "file exists" should mean that you had an old version of csi-s3. It was fixed in 0.40.3...

Noah-Huppert commented 5 months ago

Hi, "file exists" should mean that you had an old version of csi-s3. It was fixed in 0.40.3...

Ah okay. When I reinstalled k8s-csi-s3 I upgraded from v0.40.1 to v0.41.0 so I imagine that fixed it. Thanks.