search

loft-sh / vcluster

vCluster - Create fully functional virtual Kubernetes clusters - Each vcluster runs inside a namespace of the underlying k8s cluster. It's cheaper than creating separate full-blown clusters and it offers better multi-tenancy and isolation than regular namespaces.
https://www.vcluster.com
Apache License 2.0
6.32k stars 403 forks source link

setup/controller_context.go:225 couldn't find virtual cluster kube-config #1704

Closed Rockyjr-git closed 2 months ago

Rockyjr-git commented 5 months ago

What happened?

vCluster Pods are not running, Deployed in the Anthos Bare Metal Cluster using Helm Chart.

kubectl get pods -n vcluster-dev
NAME             READY   STATUS    RESTARTS      AGE
vcluster-dev-0   0/1     Running   1 (20m ago)   53m

Below are the logs of vcluster Pod.

setup/initialize.go:88  failed to find IPv6 service CIDR, will use IPv4 service CIDR. Error details: couldn't find host cluster Service CIDR ("Service "test-service-m2p76" is invalid: spec.clusterIPs[0]: Invalid value: []string{"2001:DB8::1"}: IPv6 is not configured on this cluster")        {"component": "vcluster-pro"}
2024-04-18 07:54:18     INFO    setup/controller_context.go:225 couldn't find virtual cluster kube-config, will retry in 1 seconds      {"component": "vcluster-pro"}
2024-04-18 07:54:19     INFO    setup/controller_context.go:225 couldn't find virtual cluster kube-config, will retry in 1 seconds      {"component": "vcluster-pro"}

What did you expect to happen?

vcluster and core dns pods are expected to run.

How can we reproduce it (as minimally and precisely as possible)?

Used to deploy k8s VMs in the vCluster.

Anything else we need to know?

No response

Host cluster Kubernetes version

Client Version: v1.27.3 Kustomize Version: v5.0.1 Server Version: v1.28.5-gke.1200

Host cluster Kubernetes distribution

``` # Anthos on Bare Metal Cluster ```

vlcuster version

```console $ vcluster --version # v0.19.4 ```

Vcluster Kubernetes distribution(k3s(default)), k8s, k0s)

``` # Tried with k3s and k8s. ```

OS and Arch

``` OS: ubuntu 20. 04 .6 LTS ( Focal Fossa ) Arch: ```
FabianKramm commented 5 months ago

@Rockyjr-git thanks for creating this issue! Is this a IPv6 only cluster?

Rockyjr-git commented 5 months ago

Yes, it is.

janlauber commented 4 months ago

We don't have the problem with k0sproject/k0s:v1.28.8-k0s.0 but when upgrading to k0sproject/k0s:v1.29.4-k0s.0 the following logs show up in the syncer (0.18.1 or 0.19.5) container:

2024-05-18 17:26:21 INFO    setup/initialize.go:86  k0s config secret detected, syncer will ensure that it contains service CIDR    {"component": "vcluster"}
2024-05-18 17:26:21 INFO    servicecidr/servicecidr.go:117  failed to find IPv6 service CIDR, will use IPv4 service CIDR. Error details: couldn't find host cluster Service CIDR ("Service "test-service-jmp96" is invalid: spec.clusterIPs[0]: Invalid value: []string{"2001:DB8::1"}: IPv6 is not configured on this cluster")    {"component": "vcluster"}
2024-05-18 17:26:21 INFO    k0s/k0s.go:68   Starting k0s    {"component": "vcluster", "args": "/k0s-binary/k0s controller --config=/etc/k0s/config.yaml --data-dir=/data/k0s --status-socket=/run/k0s/status.sock --disable-components=konnectivity-server,kube-scheduler,csr-approver,kube-proxy,coredns,network-provider,helm,metrics-server,kubelet-config"}
2024-05-18 17:26:21 INFO    setup/controller_context.go:221 couldn't retrieve virtual cluster version (Get "https://localhost:6443/version": dial tcp [::1]:6443: connect: connection refused), will retry in 1 seconds {"component": "vcluster"}
2024-05-18 17:26:21 INFO    setup/controller_context.go:221 couldn't retrieve virtual cluster version (Get "https://localhost:6443/version": dial tcp [::1]:6443: connect: connection refused), will retry in 1 seconds {"component": "vcluster"}
2024-05-18 17:26:21 INFO    loghelper/klog.go:24    Error: unknown component kubelet-config {"component": "vcluster", "component": "k0s"}
2024-05-18 17:26:21 INFO    setup/initialize.go:107 Error running k0s: exit status 1    {"component": "vcluster"}
2024-05-18 17:26:49 INFO    setup/initialize.go:86  k0s config secret detected, syncer will ensure that it contains service CIDR    {"component": "vcluster"}
2024-05-18 17:26:49 INFO    servicecidr/servicecidr.go:117  failed to find IPv6 service CIDR, will use IPv4 service CIDR. Error details: couldn't find host cluster Service CIDR ("Service "test-service-zv7pg" is invalid: spec.clusterIPs[0]: Invalid value: []string{"2001:DB8::1"}: IPv6 is not configured on this cluster")    {"component": "vcluster"}
2024-05-18 17:26:49 INFO    k0s/k0s.go:68   Starting k0s    {"component": "vcluster", "args": "/k0s-binary/k0s controller --config=/etc/k0s/config.yaml --data-dir=/data/k0s --status-socket=/run/k0s/status.sock --disable-components=konnectivity-server,kube-scheduler,csr-approver,kube-proxy,coredns,network-provider,helm,metrics-server,kubelet-config"}
2024-05-18 17:26:49 INFO    setup/controller_context.go:221 couldn't retrieve virtual cluster version (Get "https://localhost:6443/version": dial tcp [::1]:6443: connect: connection refused), will retry in 1 seconds {"component": "vcluster"}
2024-05-18 17:26:49 INFO    setup/controller_context.go:221 couldn't retrieve virtual cluster version (Get "https://localhost:6443/version": dial tcp [::1]:6443: connect: connection refused), will retry in 1 seconds {"component": "vcluster"}
2024-05-18 17:26:49 INFO    loghelper/klog.go:24    Error: unknown component kubelet-config {"component": "vcluster", "component": "k0s"}
2024-05-18 17:26:49 INFO    setup/initialize.go:107 Error running k0s: exit status 1    {"component": "vcluster"}
2024-05-18 17:27:33 INFO    setup/initialize.go:86  k0s config secret detected, syncer will ensure that it contains service CIDR    {"component": "vcluster"}
2024-05-18 17:27:33 INFO    servicecidr/servicecidr.go:117  failed to find IPv6 service CIDR, will use IPv4 service CIDR. Error details: couldn't find host cluster Service CIDR ("Service "test-service-ftlgh" is invalid: spec.clusterIPs[0]: Invalid value: []string{"2001:DB8::1"}: IPv6 is not configured on this cluster")    {"component": "vcluster"}
2024-05-18 17:27:33 INFO    k0s/k0s.go:68   Starting k0s    {"component": "vcluster", "args": "/k0s-binary/k0s controller --config=/etc/k0s/config.yaml --data-dir=/data/k0s --status-socket=/run/k0s/status.sock --disable-components=konnectivity-server,kube-scheduler,csr-approver,kube-proxy,coredns,network-provider,helm,metrics-server,kubelet-config"}
2024-05-18 17:27:33 INFO    setup/controller_context.go:221 couldn't retrieve virtual cluster version (Get "https://localhost:6443/version": dial tcp [::1]:6443: connect: connection refused), will retry in 1 seconds {"component": "vcluster"}
2024-05-18 17:27:33 INFO    setup/controller_context.go:221 couldn't retrieve virtual cluster version (Get "https://localhost:6443/version": dial tcp [::1]:6443: connect: connection refused), will retry in 1 seconds {"component": "vcluster"}
2024-05-18 17:27:33 INFO    loghelper/klog.go:24    Error: unknown component kubelet-config {"component": "vcluster", "component": "k0s"}
2024-05-18 17:27:33 INFO    setup/initialize.go:107 Error running k0s: exit status 1    {"component": "vcluster"}
janekbaraniewski commented 3 months ago

Hey @janlauber @Rockyjr-git ! Are you still experiencing this issue? If so, could you provide more details of your configuration? vcluster.yaml file used for installation would be a great start, also your network configuration and anthos deployment details.