vCluster PVCs stuck in Pending state

[uover822]

What happened?

Hi All :-) I'm having a similar issue to https://github.com/loft-sh/vcluster/issues/145, as my vCluster PVCs seem stuck in Pending, launching from k3d, which works fine.

My claim describes as follows:

$ k describe pvc/msr-claim
Name:          msr-claim
Namespace:     default
StorageClass:  msr-sc
Status:        Pending
Volume:        msr-volume
Labels:        <none>
Annotations:   <none>
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      0
Access Modes:  
VolumeMode:    Filesystem
Used By:       store-897d4c4d4-jvdjd
               web-574966fcbd-bvbwr
Events:        <none>

And my values file updates are here:

$ git diff values.yaml 
diff --git a/chart/values.yaml b/chart/values.yaml
index 57cd616a..2796635b 100644
--- a/chart/values.yaml
+++ b/chart/values.yaml
@@ -34,9 +34,9 @@ sync:
     serviceAccounts:
       enabled: false
     storageClasses:
-      enabled: false
+      enabled: true
     persistentVolumes:
-      enabled: false
+      enabled: true

   fromHost:
     events:
@@ -50,7 +50,7 @@ sync:
     ingressClasses:
       enabled: false
     storageClasses:
-      enabled: false
+      enabled: true
     nodes:
       enabled: false
       syncBackChanges: false
@@ -320,12 +320,12 @@ controlPlane:
         enabled: auto
         # Defines if the PVC should get automatically deleted when the StatefulSet is deleted. Can be either Delete or Retain
         retentionPolicy: Retain
-        size: 5Gi
-        storageClass: ""
+        size: 3Gi
+        storageClass: "msr-sc"
         accessModes: [ "ReadWriteOnce" ]
       volumeClaimTemplates: []
       addVolumeMounts: []
-      addVolumes: []
+      addVolumes: [ "msr-volume" ]

     scheduling:
       podManagementPolicy: Parallel
@@ -337,11 +337,11 @@ controlPlane:

     probes:
       livenessProbe:
-        enabled: true
+        enabled: false
       readinessProbe:
-        enabled: true
+        enabled: false
       startupProbe:
-        enabled: true
+        enabled: false

   serviceMonitor:
     enabled: false

My storage class seems ok:

$ k describe sc/msr-sc
Name:            msr-sc
IsDefaultClass:  No
Annotations:     kubectl.kubernetes.io/last-applied-configuration={"apiVersion":"storage.k8s.io/v1","kind":"StorageClass","metadata":{"annotations":{},"name":"msr-sc"},"provisioner":"kubernetes.io/no-provisioner","volumeBindingMode":"WaitForFirstConsumer"}

Provisioner:           kubernetes.io/no-provisioner
Parameters:            <none>
AllowVolumeExpansion:  <unset>
MountOptions:          <none>
ReclaimPolicy:         Delete
VolumeBindingMode:     WaitForFirstConsumer
Events:                <none>

As does my persistent volume:

$ k describe pv/msr-volume
Name:            msr-volume
Labels:          vcluster.loft.sh/fake-pv=true
Annotations:     kubernetes.io/createdby: fake-pv-provisioner
                 pv.kubernetes.io/bound-by-controller: true
                 pv.kubernetes.io/provisioned-by: fake-pv-provisioner
Finalizers:      [kubernetes.io/pv-protection]
StorageClass:    msr-sc
Status:          Bound
Claim:           default/msr-claim
Reclaim Policy:  Delete
Access Modes:    RWX
VolumeMode:      Filesystem
Capacity:        3Gi
Node Affinity:   <none>
Message:         
Source:
    Type:       FlexVolume (a generic volume resource that is provisioned/attached using an exec based plugin)
    Driver:     fake
    FSType:     
    SecretRef:  nil
    ReadOnly:   false
    Options:    map[]
Events:         <none>

Has anyone seen similar/ have particular thoughts/ ideas on this? I've also tried syncing with combinations of k3d resources w/o success.

Thanks :-)

What did you expect to happen?

I'm anticipating PVCs reaching Bound state and Pods reaching Running state.

How can we reproduce it (as minimally and precisely as possible)?

My storage class/ volume/ claim definitions are here:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: msr-sc
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer

apiVersion: v1
kind: PersistentVolume
metadata:
  name: msr-volume
  labels:
    type: local
spec:
  capacity:
    storage: 3Gi
  accessModes:
  - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  storageClassName: msr-sc
  hostPath:
    path: /opt/data

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: msr-claim
spec:
  volumeName: msr-volume
  accessModes:
  - ReadWriteMany
  storageClassName: msr-sc
  resources:
    requests:
      storage: 3Gi

Anything else we need to know?

Thanks :-)

Host cluster Kubernetes version

```console $ kubectl version # paste output here $ k version Client Version: v1.29.2 Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3 Server Version: v1.27.4+k3s1 ```

Host cluster Kubernetes distribution

``` # Write here $ k3d version k3d version v5.6.0 k3s version v1.27.4-k3s1 (default) ```

vlcuster version

```console $ vcluster --version # paste output here $ vc version vcluster version 0.19.5 ```

Vcluster Kubernetes distribution(k3s(default)), k8s, k0s)

``` # Write here $ k3d version k3d version v5.6.0 k3s version v1.27.4-k3s1 (default) ```

OS and Arch

``` OS: Darwin/ Mojave Arch: amd64 ```

[heiko-braun]

Hi @uover822 welcome and thanks for reporting the problem.

Can you elaborate on "launching from k3d, which works fine"?

Does it work with some vCluster distros, but not others?

[uover822]

Thanks, @heiko-braun I'm sorry for this confusion - meaning if I configure a k3d cluster directly (no vCluster), it works. Also, I haven't yet tried with earlier vCluster versions.

[neogopher]

Hi @uover822 :wave: ,

The format of values.yaml that you have used is the new format and will only work starting v0.20.0. Since, you are using v0.19.5, you may want to refer to the format from here and set the values appropriately.

Do try that out and let us know if you face any issues still.

[uover822]

Thanks for this tip, @neogopher :-) I'll check it out.

[neogopher]

Hi @uover822 ,

Closing this issue for now, but feel free to reopen in case the issue persists or if you have any further questions on this.

[uover822]

Thanks @neogopher, @heiko-braun :-) A few details below, for future reference.

After adopting the v0.19.5 configuration as suggested and customizing the following values:

   persistentvolumes:
-    enabled: false
+    enabled: true
   storageclasses:
-    enabled: false
+    enabled: true

I created the base storage class, persistent volume, and pv claim in the parent k3d/ k3s cluster, then launched my vcluster and created the inheriting storage class, persistent volume, and pv claim, and the pv claim now binds, like:

$ k get pvc
Handling connection for 10731
NAME        STATUS   VOLUME       CAPACITY   ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
msr-claim   Bound    msr-volume   3Gi        RWX            msr-sc         <unset>                 33m

Thanks for the great work/ assistance/ tips :-)