k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.VolumeAttachment: the server could not find the requested resource

[joedborg]

$ kubectl logs pod/csi-attacher-s3-0 -n kube-system
I0425 17:50:09.496327       1 reflector.go:188] Listing and watching *v1beta1.VolumeAttachment from k8s.io/client-go/informers/factory.go:135
E0425 17:50:09.502791       1 reflector.go:156] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.VolumeAttachment: the server could not find the requested resource
I0425 17:50:10.502928       1 reflector.go:188] Listing and watching *v1beta1.VolumeAttachment from k8s.io/client-go/informers/factory.go:135
E0425 17:50:10.505112       1 reflector.go:156] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.VolumeAttachment: the server could not find the requested resource
I0425 17:50:11.505270       1 reflector.go:188] Listing and watching *v1beta1.VolumeAttachment from k8s.io/client-go/informers/factory.go:135
E0425 17:50:11.513937       1 reflector.go:156] k8s.io/client-go/informers/factory.go:135: Failed to list *v1beta1.VolumeAttachment: the server could not find the requested resource

This means that the test pod won't start.

$ kubectl describe pod/csi-s3-test-nginx
Name:         csi-s3-test-nginx
Namespace:    default
Priority:     0
Node:         atlas/192.168.0.254
Start Time:   Mon, 25 Apr 2022 13:41:06 -0400
Labels:       <none>
Annotations:  <none>
Status:       Pending
IP:
IPs:          <none>
Containers:
  csi-s3-test-nginx:
    Container ID:
    Image:          nginx
    Image ID:
    Port:           <none>
    Host Port:      <none>
    State:          Waiting
      Reason:       ContainerCreating
    Ready:          False
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/lib/www/html from webroot (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-mt78j (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  webroot:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  csi-s3-pvc
    ReadOnly:   false
  kube-api-access-mt78j:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason              Age                 From                     Message
  ----     ------              ----                ----                     -------
  Warning  FailedMount         80s (x4 over 8m6s)  kubelet                  Unable to attach or mount volumes: unmounted volumes=[webroot], unattached volumes=[webroot kube-api-access-mt78j]: timed out waiting for the condition
  Warning  FailedAttachVolume  2s (x5 over 8m9s)   attachdetach-controller  AttachVolume.Attach failed for volume "pvc-0050921d-b7f2-4158-aab9-118231645848" : Attach timeout for volume pvc-0050921d-b7f2-4158-aab9-118231645848

The provisioner has worked and the bucket shows on my dashboard.

[talha0324]

Running into the same issue Any fix for this?

[liyoujunlizi]

me too.

[talha0324]

https://github.com/ctrox/csi-s3/pull/70/files

This fixes the issue for now

[liyoujunlizi]

Thank you Talha so Much, it works. And I actually use k8s v1.22.

[joedborg]

@talha0324 @liyoujunlizi

That fixed the attacher issue, but I get

Warning  FailedMount             5s (x9 over 2m13s)  kubelet                  MountVolume.MountDevice failed for volume "pvc-de09f6fb-2280-4336-b482-e76fac582301" : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name ch.ctrox.csi.s3-driver not found in the list of registered CSI drivers

on the test pod.

[talha0324]

@joedborg did you followed the install guide? seems like the drivers are not running properly or missing.

verify driver pods + storageClass

[joedborg]

@talha0324 I did:

pod/csi-attacher-s3-0                         1/1     Running   0              30m
pod/csi-provisioner-s3-0                      2/2     Running   0              30m
pod/csi-s3-9r4n7                              2/2     Running   0              30m

NAME     PROVISIONER              RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
csi-s3   ch.ctrox.csi.s3-driver   Delete          Immediate           false                  30m

[talha0324]

What namespace are you using?

[joedborg]

kube-system

[qtvhao]

https://github.com/ctrox/csi-s3/pull/70/files

This fixes the issue for now

It works, you saved my day.