search

kubernetes-retired / cluster-api-provider-nested

Cluster API Provider for Nested Clusters
Apache License 2.0
301 stars 67 forks source link

The manually created storageclass is deleted by virtual cluster #176

Closed jinsongo closed 3 years ago

jinsongo commented 3 years ago

What steps did you take and what happened: [A clear and concise description on how to REPRODUCE the bug.]

I created a most simple storage class in virtual cluster by the following yaml:

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: local-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
# oc get sc
NAME            PROVISIONER                    AGE
local-storage   kubernetes.io/no-provisioner   8s

After about 1 minute

# oc get sc
No resources found in default namespace.

What did you expect to happen: The storage is available there.

Anything else you would like to add: [Miscellaneous information that will assist in solving the issue.]

I also tried other type storage class, such as local-path-storageclass, nfs-storageclass, the problem is same.

But if I created the same type and name storage class in super cluster, the problem can be resolved. Is it a bug? or it just work as design. Thanks.

Environment:

/kind bug [One or more /area label. See https://github.com/kubernetes-sigs/cluster-api-provider-nested/labels?q=area for the list of labels]

jinsongo commented 3 years ago

@Fei-Guo @christopherhein @gyliu513 If it does work as design, we need document that, otherwise we need a fix.

gyliu513 commented 3 years ago

This is also a question from me, I think the tenant cluster should able to host its own cluster scope resources, like CRD, storageclass etc. We do check log from syncer, but seems no log can help.

christopherhein commented 3 years ago

Yes and no, this is one of the areas where the tenant can't own the resource, for a storage class to work it needs to be enabled at the super cluster level so that the kubelet can properly use the backing CSI driver, thus we enforce that storage classes are a super cluster level construct and you can share them from super -> tenant by using the tenancy.x-k8s.io/super.public: true label, this is the same way that you can from your super expose specific CRDs to a tenant while the tenant still maintains their own set of known CRDs. For the CRD case check out - https://github.com/kubernetes-sigs/cluster-api-provider-nested/blob/c6995fc3645600f9652a9b99f8c3a28426a47264/virtualcluster/doc/customresource-syncer.md#crd-synchronization

The Limitations for VC are documented - https://github.com/kubernetes-sigs/cluster-api-provider-nested/tree/c6995fc3645600f9652a9b99f8c3a28426a47264/virtualcluster#limitations and under the last bullet it talks about PVs and Storage classes.