[feature request] Allow importing of cluster-scoped resources using genericSync without multiNamespaceMode enabled

[mtougeron]

Is your feature request related to a problem?

no

Which solution do you suggest?

It would be nice to be able to import additional cluster-scoped resources such as PriorityClasses or ClusterRole(Binding)s into the vcluster using genericSync without having to enabled multiNamespaceMode.

Specifically for PriorityClasses, it would be nice to either allow importing though the experimental genericSync options or to enable it as a feature similar to the fromHost like is done for StorageClasses or CSI drivers.

For ClusterRole(Binding)s I would think it would have to happen through the genericSync options (assuming there's a label selector available).

Example desired Helm values file:

experimental:
  genericSync:
    import:
      - apiVersion: rbac.authorization.k8s.io/v1
        kind: ClusterRole
        selector:
          matchLabels:
            sync-to-vcluster: "true"
      - apiVersion: rbac.authorization.k8s.io/v1
        kind: ClusterRoleBinding
        selector:
          matchLabels:
            sync-to-vcluster: "true"
      - apiVersion: scheduling.k8s.io/v1
        kind: PriorityClass

Which alternative solutions exist?

A custom plugin could be written to do this.

Additional context

In our clusters we don't give tenants the elevated RBAC to create PriorityClasses so we cannot allow the vcluster to sync PriorityClasses on the host cluster. This means that when someone tries to create a Pod with a priority class that they expect from the host cluster it doesn't exist on the vcluster.

edit: adjusted the example for clarity

[facchettos]

There is no auto-import options available.

You can make use of the plugin system to do that, a very simple plugin as shown here would work. That would solve the problem of not giving access to the host cluster's objects

[mtougeron]

You can make use of the plugin system to do that

Good call. I'll update the original post to say that.

[dee0sap]

Btw I hit a similar problem with priorityclasses about a year ago when using 0.19.0. ( See slack thread

In my case the folks that operate the host cluster set some priorityclass global-default and that caused problems for me trying to use vcluster. With no work around in place when you try to create pods in the vcluster you tend to get this error

Type Reason Age From Message

---

Warning SyncError 15m (x87 over 17h) pod-syncer Error syncing to physical cluster: pods "test-x-default-x-test-kyverno-policies" is forbidden: the integer value of priority (0) must not be provided in pod spec; priority admission controller computed 700000000 from the given PriorityClass name

Only if your pod uses a priorityclass that can be synced down to the host clsuster do you not have a problem

My workaround is below.

vcluster create $VCLUSTER_NAME --connect=false --update-current=false -n $NAMESPACE -f <(cat <<EOF
sync:
  priorityclasses:
    enabled: true
syncer:
  extraArgs:
  - --tls-san=$VCLUSTER_API_SERVER
init:    
  manifests: |-
$(kubectl get priorityclass -o=yaml | yq '.items[] | select( .globalDefault == true )' | sed 's/^/    /' )
EOF
)

Something of note... My work around doesn't seem to work with the current 0.20.x build. It works with 0.19.6 however. When I say it doesn't seem to work with 0.20.x I mean after running it through the config conversion.