search

loft-sh / vcluster

vCluster - Create fully functional virtual Kubernetes clusters - Each vcluster runs inside a namespace of the underlying k8s cluster. It's cheaper than creating separate full-blown clusters and it offers better multi-tenancy and isolation than regular namespaces.
https://www.vcluster.com
Apache License 2.0
6.92k stars 427 forks source link

vcluster create <name> blocks execution or command does not finish... #2189

Open jakoberpf opened 1 month ago

jakoberpf commented 1 month ago

What happened?

When running vcluster create <name> from a Gitlab runner the create commands does not end.

Screenshot 2024-09-30 at 19 28 56

What did you expect to happen?

The vcluster is create and the command continues or ends.

How can we reproduce it (as minimally and precisely as possible)?

Run vcluster create <name> from a Gitlab runner.

Anything else we need to know?

The same command works fine on the same cluster but run from my local machine. MacOS, vcluster installed with brew, same version.

Host cluster Kubernetes version

```console Client Version: v1.31.0 Kustomize Version: v5.4.2 Server Version: v1.28.12-eks-a18cd3a WARNING: version difference between client (1.31) and server (1.28) exceeds the supported minor version skew of +/-1 ```

vcluster version

```console vcluster version 0.20.0 ```

VCluster Config

No special config used...

FabianKramm commented 1 month ago

@jakoberpf thanks for creating this issue! The problem is that the gitlab runner doesn't have docker installed so vcluster CLI does port-forwarding here (although I agree the messaging is not clear), which blocks the execution. If you do not want to "connect" to the vcluster after creating it you could just run the create command with --connect=false to avoid blocking execution.

jakoberpf commented 1 month ago

@FabianKramm that a great explanation thanks. One more question though. I missed to also state that the vcluster is created on a EKS cluster. Being the fact that we are creating the vcluster on a remote EKS, why is docker need on the gitlab runner for the vcluster cli?

FabianKramm commented 1 month ago

@jakoberpf it's needed to start a proxy in the background to forward the connection to the vCluster. If your vCluster is reachable by other means in the gitlab runner, you can also use the --server flag in vcluster connect to not start the background proxy, another option would be to create a load balancer via --expose. All options are described here: https://vcluster.com/docs/vcluster/manage/accessing-vcluster#expose-vcluster