search

googleforgames / agones

Dedicated Game Server Hosting and Scaling for Multiplayer Games on Kubernetes
https://agones.dev
Apache License 2.0
6.09k stars 812 forks source link

Failed calling admission webhook #707

Closed bramdutch0 closed 5 years ago

bramdutch0 commented 5 years ago

I was going through the Quickstart: Create a Game Server tutorial and am getting an error when I run "kubectl create -f https://raw.githubusercontent.com/GoogleCloudPlatform/agones/release-0.9.0/examples/simple-udp/gameserver.yaml".

The error I'm getting is: Error from server (InternalError): error when creating "https://raw.githubusercontent.com/GoogleCloudPlatform/agones/release-0.9.0/examples/simple-udp/gameserver.yaml": Internal error occurred: failed calling admission webhook "mutations.stable.agones.dev": Post https://agones-controller-service.agones-system.svc:443/mutate?timeout=30s: no endpoints available for service "agones-controller-service"

Haven't been able to find anything about this error in the documentation

markmandel commented 5 years ago

Thanks for submitting this issue. We're going to need some more information please:

bramdutch0 commented 5 years ago

Thanks for the reply!

Version Client Version: version.Info{Major:"1", Minor:"11+", GitVersion:"v1.11.9-dispatcher", GitCommit:"e3f5193e8f1091a162af7e17a781e6a3129bcfd0", GitTreeState:"clean", BuildDate:"2019-03-28T18:13:46Z", GoVersion:"go1.10.8", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"11+", GitVersion:"v1.11.8-gke.6", GitCommit:"394ee507d00f15a63cef577a14026096c310698e", GitTreeState:"clean", BuildDate:"2019-03-30T19:31:43Z", GoVersion:"go1.10.8b4", Compiler:"gc", Platform:"linux/amd64"}

Cloud Provider Google cloud

Agones Version release-0.9.0

Installed with the Yaml file

Controller logs {"filename":"/home/agones/logs/agones-controller-20190416_181112.log","message":"logging to file","numbackups":99,"severity":"info","source":"main","time":"2019-04-16T18:11:12.768866791Z"} {"ctlConf":{"MinPort":7000,"MaxPort":8000,"SidecarImage":"gcr.io/agones-images/agones-sdk:0.9.0","SidecarCPURequest":"30m","SidecarCPULimit":"0","SdkServiceAccount":"agones-sdk","AlwaysPullSidecar":false,"PrometheusMetrics":true,"Stackdriver":false,"KeyFile":"/home/agones/certs/server.key","CertFile":"/home/agones/certs/server.crt","KubeConfig":"","GCPProjectID":"","NumWorkers":100,"APIServerSustainedQPS":400,"APIServerBurstQPS":500,"LogDir":"/home/agones/logs","LogSizeLimitMB":10000},"message":"starting gameServer operator...","severity":"info","source":"main","time":"2019-04-16T18:11:12.768946255Z","version":"0.9.0"} st v1alpha1.GameServer: gameservers.stable.agones.dev is forbidden: User \"system:serviceaccount:agones-system:agones-controller\" cannot list gameservers.stable.agones.dev at the cluster scope: RBAC: clusterrole.rbac.authorization.k8s.io \"agones-controller\" not found","severity":"error","time":"2019-04-16T18:11:12.864170817Z"} {"message":"agones.dev/agones/pkg/client/informers/externalversions/factory.go:117: Failed to list v1alpha1.GameServerSet: gameserversets.stable.agones.dev is forbidden: User \"system:serviceaccount:agones-system:agones-controller\" cannot list gameserversets.stable.agones.dev at the cluster scope: RBAC: clusterrole.rbac.authorization.k8s.io \"agones-controller\" not found","severity":"error","time":"2019-04-16T18:11:12.864186074Z"} {"error":"customresourcedefinitions.apiextensions.k8s.io \"fleetautoscalers.stable.agones.dev\" is forbidden: User \"system:serviceaccount:agones-system:agones-controller\" cannot get customresourcedefinitions.apiextensions.k8s.io at the cluster scope: RBAC: clusterrole.rbac.authorization.k8s.io \"agones-controller\" not found","message":"could not start runner: *fleetautoscalers.Controller","severity":"fatal","source":"main","time":"2019-04-16T18:11:12.865730852Z"}

markmandel commented 5 years ago

Looks like your RBAC permissions are not setup.

When you installed Agones - where there any errors? I wonder if your user had permissions to setup the service accounts?

bramdutch0 commented 5 years ago

When I try to rerun the command to enable RBAC permissions (>kubectl create clusterrolebinding cluster-admin-binding \ --clusterrole cluster-admin --user gcloud config get-value account)

I get this output: Your active configuration is: [cloudshell-26994] Error from server (AlreadyExists): clusterrolebindings.rbac.authorization.k8s.io "cluster-admin-binding" already exists

I ran the commands as they appeared in the install guide (https://agones.dev/site/docs/installation/) and when I run "kubectl describe --namespace agones-system pods" I get the output described in the guide. Any ideas on what I could try to fix this?

markmandel commented 5 years ago

Are you sharing this cluster with anyone else?

What happens if you re-apply the install.yaml?

bramdutch0 commented 5 years ago

For now I'm just using the cluster for personal use.

Here is the output when I tried to reapply the yaml: serviceaccount/agones-controller unchanged clusterrolebinding.rbac.authorization.k8s.io/agones-controller-access configured serviceaccount/agones-sdk unchanged rolebinding.rbac.authorization.k8s.io/agones-sdk-access unchanged customresourcedefinition.apiextensions.k8s.io/fleets.stable.agones.dev configured customresourcedefinition.apiextensions.k8s.io/fleetallocations.stable.agones.dev configured customresourcedefinition.apiextensions.k8s.io/fleetautoscalers.stable.agones.dev configured customresourcedefinition.apiextensions.k8s.io/gameservers.stable.agones.dev configured customresourcedefinition.apiextensions.k8s.io/gameserverallocations.stable.agones.dev configured customresourcedefinition.apiextensions.k8s.io/gameserversets.stable.agones.dev configured service/agones-controller-service unchanged deployment.apps/agones-controller configured deployment.apps/agones-ping unchanged service/agones-ping-http-service unchanged service/agones-ping-udp-service unchanged priorityclass.scheduling.k8s.io/agones-system configured validatingwebhookconfiguration.admissionregistration.k8s.io/agones-validation-webhook configured mutatingwebhookconfiguration.admissionregistration.k8s.io/agones-mutation-webhook configured secret/agones-manual-cert unchanged Error from server (Forbidden): error when creating "https://raw.githubusercontent.com/GoogleCloudPlatform/agones/release-0.9.0/install/yaml/install.yaml": clusterroles.rbac.authorization.k8s.io "agones-controller" is forbidden: attempt to grant extra privileges: [{[create] [] [events] [] []} {[patch] [] [events] [] []} {[create] [] [pods] [] []} {[delete] [] [pods] [] []} {[list] [] [pods] [] []} {[watch] [] [pods] [] []} {[list] [] [nodes] [] []} {[watch] [] [nodes] [] []} {[get] [apiextensions.k8s.io] [customresourcedefinitions] [] []} {[create] [stable.agones.dev] [gameservers] [] []} {[delete] [stable.agones.dev] [gameservers] [] []} {[get] [stable.agones.dev] [gameservers] [] []} {[list] [stable.agones.dev] [gameservers] [] []} {[update] [stable.agones.dev] [gameservers] [] []} {[watch] [stable.agones.dev] [gameservers] [] []} {[create] [stable.agones.dev] [gameserversets] [] []} {[delete] [stable.agones.dev] [gameserversets] [] []} {[get] [stable.agones.dev] [gameserversets] [] []} {[list] [stable.agones.dev] [gameserversets] [] []} {[update] [stable.agones.dev] [gameserversets] [] []} {[watch] [stable.agones.dev] [gameserversets] [] []} {[update] [stable.agones.dev] [gameserversets/status] [] []} {[patch] [stable.agones.dev] [gameservers] [] []} {[get] [stable.agones.dev] [fleets] [] []} {[list] [stable.agones.dev][fleets] [] []} {[update] [stable.agones.dev] [fleets] [] []} {[watch] [stable.agones.dev] [fleets] [] []} {[get] [stable.agones.dev] [fleetallocations] [] []} {[list] [stable.agones.dev] [fleetallocations] [] []} {[update] [stable.agones.dev] [fleetallocations] [] []} {[watch] [stable.agones.dev] [fleetallocations] [] []} {[get] [stable.agones.dev] [fleetautoscalers] [] []} {[list] [stable.agones.dev] [fleetautoscalers] [] []} {[update] [stable.agones.dev] [fleetautoscalers] [] []} {[watch] [stable.agones.dev] [fleetautoscalers] [] []} {[update] [stable.agones.dev] [fleets/status] [] []} {[list] [stable.agones.dev] [gameserverallocations] [] []} {[watch] [stable.agones.dev] [gameserverallocations] [] []} {[delete] [stable.agones.dev] [gameserverallocations] [] []}] user=&{ADutch56@gmail.com [system:authenticated] map[user-assertion.cloud.google.com:[AKUJVpk6Onj0CBzTa+MDm14n5RQ6USPFOvcRwDHeUchMFFGZVyXtIH4q9muSlJOnttD9P0xUoSQiD5F4nz06i1iWtvXck5kTdwyh1CvKX5SCD9o1QyGyD0Ni1SdfK+FxAk1l+r7qhS0scZR2JMMOMOhA+Uk4dIZ3Y0u3baF5IYgWwM/I/DSPknPiGfidV7MOXy32YSuVNcjbH4ezr6B52XCQkdGj306R3kBzow==]]} ownerrules=[{[create] [authorization.k8s.io] [selfsubjectaccessreviews selfsubjectrulesreviews] [] []} {[get] [] [] [] [/api /api/ /apis /apis/ /healthz /openapi /openapi/ /swagger-2.0.0.pb-v1 /swagger.json /swaggerapi /swaggerapi/ /version /version/]}] ruleResolutionErrors=[] Error from server (Forbidden): error when creating "https://raw.githubusercontent.com/GoogleCloudPlatform/agones/release-0.9.0/install/yaml/install.yaml": clusterroles.rbac.authorization.k8s.io "agones-sdk" is forbidden: attempt to grant extra privileges: [{[create] [] [events] [] []} {[list] [stable.agones.dev] [gameservers] [] []} {[update] [stable.agones.dev] [gameservers] [] []} {[watch] [stable.agones.dev] [gameservers] [] []}] user=&{ADutch56@gmail.com [system:authenticated] map[user-assertion.cloud.google.com:[AKUJVpk6Onj0CBzTa+MDm14n5RQ6USPFOvcRwDHeUchMFFGZVyXtIH4q9muSlJOnttD9P0xUoSQiD5F4nz06i1iWtvXck5kTdwyh1CvKX5SCD9o1QyGyD0Ni1SdfK+FxAk1l+r7qhS0scZR2JMMOMOhA+Uk4dIZ3Y0u3baF5IYgWwM/I/DSPknPiGfidV7MOXy32YSuVNcjbH4ezr6B52XCQkdGj306R3kBzow==]]} ownerrules=[{[create] [authorization.k8s.io] [selfsubjectaccessreviews selfsubjectrulesreviews] [] []} {[get] [] [] [] [/api /api/ /apis /apis/ /healthz /openapi /openapi/ /swagger-2.0.0.pb-v1 /swagger.json /swaggerapi /swaggerapi/ /version /version/]}] ruleResolutionErrors=[]

markmandel commented 5 years ago

Looks like there are issues with your cluster-admin role. Maybe try deleting it and re-adding it, and then try re-applying the index.yaml.

There should be no Forbidden items in the list.

bramdutch0 commented 5 years ago

It turns out there was an issue with RBAC in GKE. This post helped me troubleshoot it if anyone else runs into this problem: https://github.com/coreos/prometheus-operator/issues/357

After that I got the yaml install file to run with no errors.

markmandel commented 5 years ago

I doubt there is an issue with the simple-udp example, as use it for demos and testing quite regularly. I expect your controller wasn't up yet when you pushed the yaml file.

Sounds like you have resolved the issue :+1: I'll close the issue for now. Please feel free to reopen if you deem it necessary.

markmandel commented 5 years ago

If you are looking for the same howto in our documentation, it is here: https://agones.dev/site/docs/installation/#enabling-creation-of-rbac-resources

(We likely should split this up by provider to make things easier though)

bramdutch0 commented 5 years ago

Everything seems to be working fine with the tutorial. Thanks for your help!