Closed ptaylor10 closed 5 years ago
Please take a look at this excerpt from the prerequisites in readme and let me know if this fixes your problem:
If you are using a GKE Kubernetes cluster, you will need to grant your current Google identity cluster-admin role. Otherwise you won't be authorized to grant extra privileges to the VPA system components.
$ gcloud info | grep Account # get current google identity
Account: [myname@example.org]
$ kubectl create clusterrolebinding myname-cluster-admin-binding --clusterrole=cluster-admin --user=myname@example.org
Clusterrolebinding "myname-cluster-admin-binding" created
I did run that prior to vpa-up.sh and it looks like the clusters roles were created; however, then they are denied and forbidden access. I also given the service account Kubernetes-Cluster-Admin rights as well.
Here is the full messages I get when I run vpa-up.sh
customresourcedefinition.apiextensions.k8s.io "verticalpodautoscalers.autoscaling.k8s.io" created
customresourcedefinition.apiextensions.k8s.io "verticalpodautoscalercheckpoints.autoscaling.k8s.io" created
clusterrolebinding.rbac.authorization.k8s.io "system:metrics-reader" created
clusterrolebinding.rbac.authorization.k8s.io "system:vpa-actor" created
clusterrolebinding.rbac.authorization.k8s.io "system:vpa-checkpoint-actor" created
clusterrolebinding.rbac.authorization.k8s.io "system:vpa-updater-controllers-reader-binding" created
clusterrolebinding.rbac.authorization.k8s.io "system:vpa-evictionter-binding" created
serviceaccount "vpa-admission-controller" created
clusterrolebinding.rbac.authorization.k8s.io "system:admission-controller" created
Error from server (Forbidden): error when creating "STDIN": clusterroles.rbac.authorization.k8s.io "system:metrics-reader" is forbidden: attempt to grant extra privileges: [{[get] [metrics.k8s.io] [pods] [] []} {[list] [metrics.k8s.io] [pods] [] []}] user=&{118045764671172196470 [system:authenticated] map[user-assertion.cloud.google.com:[AK5xou+nymRus51hZrGpnKZp+wumChHD1DUAkO0V9+1+3tplSI/bcsKf9Vrmko5bjE6BdoroAgMj+UpBvPOTkN/rXsR3TW6/YDFOLTxh5tlUUjhjQKig8p1zVxfrWb8D0zGd02/vTI1b7PRSwN0LQqu3Tr2e4fuhv30Nf1SvSW0euzB+avtlzeBmzETBQYsFv58q/MNYhKQIRyuwDtMJaaxU0e3iwwB8xCeaELTWew==]]} ownerrules=[{[create] [authorization.k8s.io] [selfsubjectaccessreviews selfsubjectrulesreviews] [] []} {[get] [] [] [] [/api /api/* /apis /apis/* /healthz /openapi /openapi/* /swagger-2.0.0.pb-v1 /swagger.json /swaggerapi /swaggerapi/* /version /version/]}] ruleResolutionErrors=[]
Error from server (Forbidden): error when creating "STDIN": clusterroles.rbac.authorization.k8s.io "system:vpa-actor" is forbidden: attempt to grant extra privileges: [{[get] [] [pods] [] []} {[list] [] [pods] [] []} {[watch] [] [pods] [] []} {[get] [] [nodes] [] []} {[list] [] [nodes] [] []} {[watch] [] [nodes] [] []} {[get] [] [events] [] []} {[list] [] [events] [] []} {[watch] [] [events] [] []} {[create] [] [events] [] []} {[get] [poc.autoscaling.k8s.io] [verticalpodautoscalers] [] []} {[list] [poc.autoscaling.k8s.io] [verticalpodautoscalers] [] []} {[watch] [poc.autoscaling.k8s.io] [verticalpodautoscalers] [] []} {[patch] [poc.autoscaling.k8s.io] [verticalpodautoscalers] [] []} {[get] [autoscaling.k8s.io] [verticalpodautoscalers] [] []} {[list] [autoscaling.k8s.io] [verticalpodautoscalers] [] []} {[watch] [autoscaling.k8s.io] [verticalpodautoscalers] [] []} {[patch] [autoscaling.k8s.io] [verticalpodautoscalers] [] []}] user=&{118045764671172196470 [system:authenticated] map[user-assertion.cloud.google.com:[AK5xou+nymRus51hZrGpnKZp+wumChHD1DUAkO0V9+1+3tplSI/bcsKf9Vrmko5bjE6BdoroAgMj+UpBvPOTkN/rXsR3TW6/YDFOLTxh5tlUUjhjQKig8p1zVxfrWb8D0zGd02/vTI1b7PRSwN0LQqu3Tr2e4fuhv30Nf1SvSW0euzB+avtlzeBmzETBQYsFv58q/MNYhKQIRyuwDtMJaaxU0e3iwwB8xCeaELTWew==]]} ownerrules=[{[create] [authorization.k8s.io] [selfsubjectaccessreviews selfsubjectrulesreviews] [] []} {[get] [] [] [] [/api /api/* /apis /apis/* /healthz /openapi /openapi/* /swagger-2.0.0.pb-v1 /swagger.json /swaggerapi /swaggerapi/* /version /version/]}] ruleResolutionErrors=[]
Error from server (Forbidden): error when creating "STDIN": clusterroles.rbac.authorization.k8s.io "system:vpa-checkpoint-actor" is forbidden: attempt to grant extra privileges: [{[get] [poc.autoscaling.k8s.io] [verticalpodautoscalercheckpoints] [] []} {[list] [poc.autoscaling.k8s.io] [verticalpodautoscalercheckpoints] [] []} {[watch] [poc.autoscaling.k8s.io] [verticalpodautoscalercheckpoints] [] []} {[create] [poc.autoscaling.k8s.io] [verticalpodautoscalercheckpoints] [] []} {[patch] [poc.autoscaling.k8s.io] [verticalpodautoscalercheckpoints] [] []} {[delete] [poc.autoscaling.k8s.io] [verticalpodautoscalercheckpoints] [] []} {[get] [autoscaling.k8s.io] [verticalpodautoscalercheckpoints] [] []} {[list] [autoscaling.k8s.io] [verticalpodautoscalercheckpoints] [] []} {[watch] [autoscaling.k8s.io] [verticalpodautoscalercheckpoints] [] []} {[create] [autoscaling.k8s.io] [verticalpodautoscalercheckpoints] [] []} {[patch] [autoscaling.k8s.io] [verticalpodautoscalercheckpoints] [] []} {[delete] [autoscaling.k8s.io] [verticalpodautoscalercheckpoints] [] []} {[get] [] [namespaces] [] []} {[list] [] [namespaces] [] []}] user=&{118045764671172196470 [system:authenticated] map[user-assertion.cloud.google.com:[AK5xou+nymRus51hZrGpnKZp+wumChHD1DUAkO0V9+1+3tplSI/bcsKf9Vrmko5bjE6BdoroAgMj+UpBvPOTkN/rXsR3TW6/YDFOLTxh5tlUUjhjQKig8p1zVxfrWb8D0zGd02/vTI1b7PRSwN0LQqu3Tr2e4fuhv30Nf1SvSW0euzB+avtlzeBmzETBQYsFv58q/MNYhKQIRyuwDtMJaaxU0e3iwwB8xCeaELTWew==]]} ownerrules=[{[create] [authorization.k8s.io] [selfsubjectaccessreviews selfsubjectrulesreviews] [] []} {[get] [] [] [] [/api /api/* /apis /apis/* /healthz /openapi /openapi/* /swagger-2.0.0.pb-v1 /swagger.json /swaggerapi /swaggerapi/* /version /version/]}] ruleResolutionErrors=[]
Error from server (Forbidden): error when creating "STDIN": clusterroles.rbac.authorization.k8s.io "system:evictioner" is forbidden: attempt to grant extra privileges: [{[get] [extensions] [replicasets] [] []} {[create] [] [pods/eviction] [] []}] user=&{118045764671172196470 [system:authenticated] map[user-assertion.cloud.google.com:[AK5xou+nymRus51hZrGpnKZp+wumChHD1DUAkO0V9+1+3tplSI/bcsKf9Vrmko5bjE6BdoroAgMj+UpBvPOTkN/rXsR3TW6/YDFOLTxh5tlUUjhjQKig8p1zVxfrWb8D0zGd02/vTI1b7PRSwN0LQqu3Tr2e4fuhv30Nf1SvSW0euzB+avtlzeBmzETBQYsFv58q/MNYhKQIRyuwDtMJaaxU0e3iwwB8xCeaELTWew==]]} ownerrules=[{[create] [authorization.k8s.io] [selfsubjectaccessreviews selfsubjectrulesreviews] [] []} {[get] [] [] [] [/api /api/* /apis /apis/* /healthz /openapi /openapi/* /swagger-2.0.0.pb-v1 /swagger.json /swaggerapi /swaggerapi/* /version /version/]}] ruleResolutionErrors=[]
Error from server (Forbidden): error when creating "STDIN": clusterroles.rbac.authorization.k8s.io "system:controllers-reader" is forbidden: attempt to grant extra privileges: [{[get] [] [replicationcontrollers] [] []} {[list] [] [replicationcontrollers] [] []} {[watch] [] [replicationcontrollers] [] []} {[get] [apps] [statefulsets] [] []} {[list] [apps] [statefulsets] [] []} {[watch] [apps] [statefulsets] [] []} {[get] [apps] [replicasets] [] []} {[list] [apps] [replicasets] [] []} {[watch] [apps] [replicasets] [] []}] user=&{118045764671172196470 [system:authenticated] map[user-assertion.cloud.google.com:[AK5xou+nymRus51hZrGpnKZp+wumChHD1DUAkO0V9+1+3tplSI/bcsKf9Vrmko5bjE6BdoroAgMj+UpBvPOTkN/rXsR3TW6/YDFOLTxh5tlUUjhjQKig8p1zVxfrWb8D0zGd02/vTI1b7PRSwN0LQqu3Tr2e4fuhv30Nf1SvSW0euzB+avtlzeBmzETBQYsFv58q/MNYhKQIRyuwDtMJaaxU0e3iwwB8xCeaELTWew==]]} ownerrules=[{[create] [authorization.k8s.io] [selfsubjectaccessreviews selfsubjectrulesreviews] [] []} {[get] [] [] [] [/api /api/* /apis /apis/* /healthz /openapi /openapi/* /swagger-2.0.0.pb-v1 /swagger.json /swaggerapi /swaggerapi/* /version /version/]}] ruleResolutionErrors=[]
Error from server (Forbidden): error when creating "STDIN": clusterroles.rbac.authorization.k8s.io "system:admission-controller" is forbidden: attempt to grant extra privileges: [{[get] [] [pods] [] []} {[list] [] [pods] [] []} {[watch] [] [pods] [] []} {[get] [] [configmaps] [] []} {[list] [] [configmaps] [] []} {[watch] [] [configmaps] [] []} {[get] [] [nodes] [] []} {[list] [] [nodes] [] []} {[watch] [] [nodes] [] []} {[create] [admissionregistration.k8s.io] [mutatingwebhookconfigurations] [] []} {[delete] [admissionregistration.k8s.io] [mutatingwebhookconfigurations] [] []} {[get] [admissionregistration.k8s.io] [mutatingwebhookconfigurations] [] []} {[list] [admissionregistration.k8s.io] [mutatingwebhookconfigurations] [] []} {[get] [poc.autoscaling.k8s.io] [verticalpodautoscalers] [] []} {[list] [poc.autoscaling.k8s.io] [verticalpodautoscalers] [] []} {[watch] [poc.autoscaling.k8s.io] [verticalpodautoscalers] [] []} {[get] [autoscaling.k8s.io] [verticalpodautoscalers] [] []} {[list] [autoscaling.k8s.io] [verticalpodautoscalers] [] []} {[watch] [autoscaling.k8s.io] [verticalpodautoscalers] [] []}] user=&{118045764671172196470 [system:authenticated] map[user-assertion.cloud.google.com:[AK5xou+nymRus51hZrGpnKZp+wumChHD1DUAkO0V9+1+3tplSI/bcsKf9Vrmko5bjE6BdoroAgMj+UpBvPOTkN/rXsR3TW6/YDFOLTxh5tlUUjhjQKig8p1zVxfrWb8D0zGd02/vTI1b7PRSwN0LQqu3Tr2e4fuhv30Nf1SvSW0euzB+avtlzeBmzETBQYsFv58q/MNYhKQIRyuwDtMJaaxU0e3iwwB8xCeaELTWew==]]} ownerrules=[{[create] [authorization.k8s.io] [selfsubjectaccessreviews selfsubjectrulesreviews] [] []} {[get] [] [] [] [/api /api/* /apis /apis/* /healthz /openapi /openapi/* /swagger-2.0.0.pb-v1 /swagger.json /swaggerapi /swaggerapi/* /version /version/]}] ruleResolutionErrors=[]
serviceaccount "vpa-updater" created
deployment.extensions "vpa-updater" created
serviceaccount "vpa-recommender" created
deployment.extensions "vpa-recommender" created
Generating certs for the VPA Admission Controller in /tmp/vpa-certs.
Generating RSA private key, 2048 bit long modulus
....................+++++
....+++++
e is 65537 (0x010001)
Generating RSA private key, 2048 bit long modulus
...........+++++
.....................................................................................................................................................................................................+++++
e is 65537 (0x010001)
Signature ok
subject=CN = vpa-webhook.kube-system.svc
Getting CA Private Key
Uploading certs to the cluster.
secret "vpa-tls-certs" created
Deleting /tmp/vpa-certs.
deployment.extensions "vpa-admission-controller" created
service "vpa-webhook" created
I figured out the issue. It looks like it was using a different principal Email to make these changes. Instead of my service account email it was using it's uniqueID.
I had to do the following (in case someone else runs into this as well):
gcloud iam service-accounts describe [myname@example.org] | grep uniqueId
I am trying out Vertical Pod Autoscaler and haven't had any luck with deploying it. I am using a new Cluster on Google Cloud with the following kubernetes versions:
However, I am seeing these as an errors and it looks like the vpa isn't working correctly as well:
I also noticed these log errors on the vpa pod: