search

kubestellar / kubeflex

A flexible and scalable platform for running Kubernetes control plane APIs.
Apache License 2.0
46 stars 13 forks source link

bug: accessing kubeflex controlplane from within a pod #89

Closed effi-ofer closed 11 months ago

effi-ofer commented 11 months ago

Describe the bug

I can use your help in accessing a kubeflex controlplane from within a pod. This is what I have done: (1) kflex init --create-kind on a kind cluster which created the kubeflex kind cluster. I then created a pod from within kubeflex core cluster (i.e. the kind cluster named kubeflex created in step 1) and issued (2) kflex create cp1 from within. I can now access cp1 externally using the admin-kubeconfig secret. But I am not able to access cp1 from within the pod running in the kubeflex core cluster. Using the cm-kubeconfig points to the same url as admin-kubeconfig and that's not visible from the pod. I tried using the ingress-nginx that kubeflex created and that didn't help either. If you happen to have 30min to sit with me on this, I would greatly appreciate it. (edited)

Steps To Reproduce

See above.

Expected Behavior

See above.

Additional Context

No response

pdettori commented 11 months ago

Thanks @effi-ofer for opening the issue. I can replicate the issue and see that the generated URL in the cm-kubeconfig is indeed server: https://cp1.localtest.me:9443.

effi-ofer commented 11 months ago

Thanks @pdettori. If you can tell me what's the proper url, I'll be able to continue while you resolve the code.

pdettori commented 11 months ago

hi @effi-ofer, for the time being you may use the admin kubeconfig with URL https://<control-plane-name>.<control-plane-name>-system.svc.cluster.local:9444 e.g. https://cp2.cp2-system.svc.cluster.local:9444

pdettori commented 11 months ago

I have also realized that the cm-kubeconfig has just the set of permissions required by the subset of controllers run by the kube controller manager. For this reason, there are actually 3 different kubeconfigs:

  1. admin kubeconfig with external URL
  2. admin kubeconfig with cluster internal URL
  3. Controller Manager kubeconfig with cluster internal URL. For your usage you'd actually need 2. One option is to add another key to the secret admin-kubeconfig for that, would that work for your use case?
effi-ofer commented 11 months ago

That would work for me. Also, I tried your url suggestion and it worked. Thank you again.