search

GoogleCloudPlatform / metacontroller

Lightweight Kubernetes controllers as a service
https://metacontroller.app/
Apache License 2.0
792 stars 105 forks source link

--client-config-path option does not work #178

Closed pragyamehta closed 5 years ago

pragyamehta commented 5 years ago

I am trying out the changes introduced in https://github.com/GoogleCloudPlatform/metacontroller/pull/113/commits/db0ecdf660505fdea3ef549970aa70c6472a9ed3 for the issue https://github.com/GoogleCloudPlatform/metacontroller/issues/111

Suppose you have 2 clusters - "metacluster" and "usercluster". Below are the steps I am following :

  1. I am installing metacontroller using the steps mentioned in https://metacontroller.app/guide/install/ in the metacluster
  2. Next, I upload the kubeconfig of usercluster as a configmap in the metacluster.
  3. Then, I attach the configmap as a volume to the statefulset of metacontroller and also update the metacontroller container command to add --client-config-path and pass the kubeconfig path to it in metacluster.
  4. Next, I create a CRD "hello-world" in usercluster
  5. I create a compositecontroller for this CRD in metacluster and deploy the webhook in metacluster
  6. With this setup ready, I now try to create a k8s object of type "hello-world" in usercluster

Expectation: Child resources of "hello-world" CRD should spin up in usercluster Observation: No child resources spin up. I see the below errors in metacontoller pod in metacluster : metacontroller.app/client/generated/informer/externalversions/factory.go:117: Failed to list *v1alpha1.ControllerRevision: the server could not find the requested resource (get controllerrevisions.metacontroller.k8s.io) metacontroller.app/client/generated/informer/externalversions/factory.go:117: Failed to list *v1alpha1.DecoratorController: the server could not find the requested resource (get decoratorcontrollers.metacontroller.k8s.io) metacontroller.app/client/generated/informer/externalversions/factory.go:117: Failed to list *v1alpha1.CompositeController: the server could not find the requested resource (get compositecontrollers.metacontroller.k8s.io)

I can see from the logs that the metacontroller pod in metacluster is looking for controllerrevision, decoratorcontroller and compositecontroller in the usercluster instead of the metacluster. From my understanding, it should be looking for it in the metacluster instead.

Can anyone explain how to get this scenario to work? Is my understanding of the change correct?

pragyamehta commented 5 years ago

I think I found my mistake. The composite controller CRD and hello-world composite controller need to deployed to the usercluster for this to work.