search

loft-sh / vcluster

vCluster - Create fully functional virtual Kubernetes clusters - Each vcluster runs inside a namespace of the underlying k8s cluster. It's cheaper than creating separate full-blown clusters and it offers better multi-tenancy and isolation than regular namespaces.
https://www.vcluster.com
Apache License 2.0
6.68k stars 422 forks source link

feat: vcluster check (pre-flight checklist for installation) #42

Open kostis-codefresh opened 3 years ago

kostis-codefresh commented 3 years ago

I am trying to use vcluster and according to the home page "As long as you can create a deployment inside a single namespace, you will be able to create a virtual cluster and become admin of this virtual cluster."

I have a limited account in Okteto.com where I have access only to a namespace but not to the whole cluster. While a simple deployment works, vcluster doesn't install.

~ [1] $ kubectl create deployment nginx --image=nginx -n virtual-kostis-codefresh
deployment.apps/nginx created

~ $ vcluster create sample1 -n virtual-kostis-codefresh
[info]   execute command: helm upgrade sample1 vcluster --repo https://charts.loft.sh --version 0.3.0-beta.0 --kubeconfig /tmp/416075950 --namespace virtual-kostis-codefresh --install --repository-config='' --values /tmp/271665205
[fatal]  error executing helm upgrade sample1 vcluster --repo https://charts.loft.sh --version 0.3.0-beta.0 --kubeconfig /tmp/416075950 --namespace virtual-kostis-codefresh --install --repository-config='' --values /tmp/271665205: Release "sample1" does not exist. Installing it now.
Error: admission webhook "role.webhook.okteto.com" denied the request: rule 0 for role 'sample1' is not allowed: {Verbs:[*] APIGroups:[] Resources:[configmaps secrets services services/proxy pods pods/proxy pods/attach pods/portforward pods/exec pods/log events endpoints persistentvolumeclaims] ResourceNames:[] NonResourceURLs:[]}

I think it would be great if vcluster had a "check" option that mentions if it can be installed in the cluster and reports on what is needed and what is missing. Other tools have a similar feature as well. See for example https://linkerd.io/2.10/reference/cli/check/

FabianKramm commented 3 years ago

@kostis-codefresh thanks for creating this issue! Yes we can think about adding this to make deployment of vcluster more easy

rio commented 3 years ago

This would definitely help with deploying when having only access to the admin role in a namespace. This is what currently happens:

~ $ vcluster create vcluster-1
[info]   execute command: helm upgrade vcluster-1 vcluster --repo https://charts.loft.sh --version 0.3.0 --kubeconfig /tmp/247832214 --namespace vcluster --install --repository-config='' --values /tmp/285100285
[fatal]  error executing helm upgrade vcluster-1 vcluster --repo https://charts.loft.sh --version 0.3.0 --kubeconfig /tmp/247832214 --namespace vcluster --install --repository-config='' --values /tmp/285100285: Error: UPGRADE FAILED: failed to create resource: roles.rbac.authorization.k8s.io "vcluster-1" is forbidden: user "system:serviceaccount:vcluster:default" (groups=["system:serviceaccounts" "system:serviceaccounts:vcluster" "system:authenticated"]) is attempting to grant RBAC permissions not currently held:
{APIGroups:[""], Resources:["configmaps"], Verbs:["*"]}
{APIGroups:[""], Resources:["endpoints"], Verbs:["*"]}
{APIGroups:[""], Resources:["events"], Verbs:["*"]}
{APIGroups:[""], Resources:["persistentvolumeclaims"], Verbs:["*"]}
{APIGroups:[""], Resources:["pods"], Verbs:["*"]}
{APIGroups:[""], Resources:["pods/attach"], Verbs:["*"]}
{APIGroups:[""], Resources:["pods/exec"], Verbs:["*"]}
{APIGroups:[""], Resources:["pods/log"], Verbs:["*"]}
{APIGroups:[""], Resources:["pods/portforward"], Verbs:["*"]}
{APIGroups:[""], Resources:["pods/proxy"], Verbs:["*"]}
{APIGroups:[""], Resources:["secrets"], Verbs:["*"]}
{APIGroups:[""], Resources:["services"], Verbs:["*"]}
{APIGroups:[""], Resources:["services/proxy"], Verbs:["*"]}
{APIGroups:["networking.k8s.io"], Resources:["ingresses"], Verbs:["*"]}
rio commented 3 years ago

For completeness this is the admin clusterrole in kind v0.11.1

Name:         admin
Labels:       kubernetes.io/bootstrapping=rbac-defaults
Annotations:  rbac.authorization.kubernetes.io/autoupdate: true
PolicyRule:
  Resources                                       Non-Resource URLs  Resource Names  Verbs
  ---------                                       -----------------  --------------  -----
  rolebindings.rbac.authorization.k8s.io          []                 []              [create delete deletecollection get list patch update watch]
  roles.rbac.authorization.k8s.io                 []                 []              [create delete deletecollection get list patch update watch]
  configmaps                                      []                 []              [create delete deletecollection patch update get list watch]
  endpoints                                       []                 []              [create delete deletecollection patch update get list watch]
  persistentvolumeclaims                          []                 []              [create delete deletecollection patch update get list watch]
  pods                                            []                 []              [create delete deletecollection patch update get list watch]
  replicationcontrollers/scale                    []                 []              [create delete deletecollection patch update get list watch]
  replicationcontrollers                          []                 []              [create delete deletecollection patch update get list watch]
  services                                        []                 []              [create delete deletecollection patch update get list watch]
  daemonsets.apps                                 []                 []              [create delete deletecollection patch update get list watch]
  deployments.apps/scale                          []                 []              [create delete deletecollection patch update get list watch]
  deployments.apps                                []                 []              [create delete deletecollection patch update get list watch]
  replicasets.apps/scale                          []                 []              [create delete deletecollection patch update get list watch]
  replicasets.apps                                []                 []              [create delete deletecollection patch update get list watch]
  statefulsets.apps/scale                         []                 []              [create delete deletecollection patch update get list watch]
  statefulsets.apps                               []                 []              [create delete deletecollection patch update get list watch]
  horizontalpodautoscalers.autoscaling            []                 []              [create delete deletecollection patch update get list watch]
  cronjobs.batch                                  []                 []              [create delete deletecollection patch update get list watch]
  jobs.batch                                      []                 []              [create delete deletecollection patch update get list watch]
  daemonsets.extensions                           []                 []              [create delete deletecollection patch update get list watch]
  deployments.extensions/scale                    []                 []              [create delete deletecollection patch update get list watch]
  deployments.extensions                          []                 []              [create delete deletecollection patch update get list watch]
  ingresses.extensions                            []                 []              [create delete deletecollection patch update get list watch]
  networkpolicies.extensions                      []                 []              [create delete deletecollection patch update get list watch]
  replicasets.extensions/scale                    []                 []              [create delete deletecollection patch update get list watch]
  replicasets.extensions                          []                 []              [create delete deletecollection patch update get list watch]
  replicationcontrollers.extensions/scale         []                 []              [create delete deletecollection patch update get list watch]
  ingresses.networking.k8s.io                     []                 []              [create delete deletecollection patch update get list watch]
  networkpolicies.networking.k8s.io               []                 []              [create delete deletecollection patch update get list watch]
  poddisruptionbudgets.policy                     []                 []              [create delete deletecollection patch update get list watch]
  deployments.apps/rollback                       []                 []              [create delete deletecollection patch update]
  deployments.extensions/rollback                 []                 []              [create delete deletecollection patch update]
  localsubjectaccessreviews.authorization.k8s.io  []                 []              [create]
  pods/attach                                     []                 []              [get list watch create delete deletecollection patch update]
  pods/exec                                       []                 []              [get list watch create delete deletecollection patch update]
  pods/portforward                                []                 []              [get list watch create delete deletecollection patch update]
  pods/proxy                                      []                 []              [get list watch create delete deletecollection patch update]
  secrets                                         []                 []              [get list watch create delete deletecollection patch update]
  services/proxy                                  []                 []              [get list watch create delete deletecollection patch update]
  bindings                                        []                 []              [get list watch]
  events                                          []                 []              [get list watch]
  limitranges                                     []                 []              [get list watch]
  namespaces/status                               []                 []              [get list watch]
  namespaces                                      []                 []              [get list watch]
  persistentvolumeclaims/status                   []                 []              [get list watch]
  pods/log                                        []                 []              [get list watch]
  pods/status                                     []                 []              [get list watch]
  replicationcontrollers/status                   []                 []              [get list watch]
  resourcequotas/status                           []                 []              [get list watch]
  resourcequotas                                  []                 []              [get list watch]
  services/status                                 []                 []              [get list watch]
  controllerrevisions.apps                        []                 []              [get list watch]
  daemonsets.apps/status                          []                 []              [get list watch]
  deployments.apps/status                         []                 []              [get list watch]
  replicasets.apps/status                         []                 []              [get list watch]
  statefulsets.apps/status                        []                 []              [get list watch]
  horizontalpodautoscalers.autoscaling/status     []                 []              [get list watch]
  cronjobs.batch/status                           []                 []              [get list watch]
  jobs.batch/status                               []                 []              [get list watch]
  daemonsets.extensions/status                    []                 []              [get list watch]
  deployments.extensions/status                   []                 []              [get list watch]
  ingresses.extensions/status                     []                 []              [get list watch]
  replicasets.extensions/status                   []                 []              [get list watch]
  ingresses.networking.k8s.io/status              []                 []              [get list watch]
  poddisruptionbudgets.policy/status              []                 []              [get list watch]
  serviceaccounts                                 []                 []              [impersonate create delete deletecollection patch update get list watch]
FabianKramm commented 3 years ago

@kostis-codefresh we just released v0.3.1-alpha.0 that uses specific verbs and reduced rbac rights for the role. Could you check if that works for you and fixes your problem?

kostis-codefresh commented 3 years ago

With the new version I got

./vcluster-linux-amd64 create sample1 -n virtual-kostis-codefresh
[info]   execute command: helm upgrade sample1 vcluster --repo https://charts.loft.sh --version 0.3.1-alpha.0 --kubeconfig /tmp/327907676 --namespace virtual-kostis-codefresh --install --repository-config='' --values /tmp/007889675
[fatal]  error executing helm upgrade sample1 vcluster --repo https://charts.loft.sh --version 0.3.1-alpha.0 --kubeconfig /tmp/327907676 --namespace virtual-kostis-codefresh --install --repository-config='' --values /tmp/007889675: Release "sample1" does not exist. Installing it now.
Error: admission webhook "role.webhook.okteto.com" denied the request: rule 3 for role 'sample1' is not allowed: {Verbs:[get list watch] APIGroups:[] Resources:[namespaces] ResourceNames:[] NonResourceURLs:[]}
jiridanek commented 11 months ago 
$ vcluster version
vcluster version 0.18.1
$ vcluster create rhm--vcluster --namespace rhm--vcluster --create-namespace=false
20:35:38 info Create vcluster rhm--vcluster...
20:35:38 info execute command: helm upgrade rhm--vcluster /tmp/vcluster-0.18.1.tgz-1515648162 --kubeconfig /tmp/1338616499 --namespace rhm--vcluster --install --repository-config='' --values /tmp/4049134534
20:35:44 fatal error executing helm upgrade rhm--vcluster /tmp/vcluster-0.18.1.tgz-1515648162 --kubeconfig /tmp/1338616499 --namespace rhm--vcluster --install --repository-config='' --values /tmp/4049134534: Release "rhm--vcluster" does not exist. Installing it now.
Error: rendered manifests contain a resource that already exists. Unable to continue with install: could not get information about the resource Role "rhm--vcluster" in namespace "rhm--vcluster": roles.rbac.authorization.k8s.io "rhm--vcluster" is forbidden: User "jdanek" cannot get resource "roles" in API group "rbac.authorization.k8s.io" in the namespace "rhm--vcluster"

This looks different from the error @kostis-codefresh got before me.