[RFE] Add functionality to allow setting up a cluster dynamically in the backend

[joaquimrocha]

Current situation

Headlamp's backend simply reads the in-cluster configuration, or existing kubeconfig files. This means that new clusters cannot be dynamically created which prevents functionality like: loading a cluster that's defined by a plugin or allowing the user to load a kubeconfig from the UI.

Ideal future situation

We allow to dynamically set up new clusters in the backend, so we open new possibilities for cluster management in Headlamp.

Implementation options

I think that we can have an API endpoint in the backend that allows to set up new clusters by passing it the cluster's details. This endpoint should be off by default when running in-cluster, but there should be a CLI option to turn it on anyway, in case a Headlamp deployment wants to allow that.

[joaquimrocha]

The way I am thinking about this is:

We have a backend endpoint like /cluster/new. It will take a json body of the cluster config similar to the one set up in a kube config file. There is also a Headlamp own option for making this new cluster persistent (I will get there).

Once the backend gets this info, it will start a new proxy for that server. i.e. there will be a new clusters/CLUSTER_NAME endpoint waiting for redirecting API calls to the respective kubernetes server. After this step, any calls to /config will include the mentioned clusters in the respective entry (so the client may know exactly what clusters are configured).

Persistency

If the persistency option is true, then the backend will save the clusters' config (kubeconfigs) into Headlamp's own config folder, so they can be reloaded next time.

We also have this as an option and not as default behavior because 1) maybe plugins want to keep dynamically populate clusters and not let them persist next time; 2) If we allow to dynamically set up clusters in a hosted Headlamp (in-cluster), then maybe persisting the clusters is kind of complicated in this way.

Options options

We may want to have switches for whether persistency or even the whole dynamic cluster set up is allowed. I imagine that, again, for in-cluster deployments of Headlamp, we don't want this functionality available, at least not by default. So besides making this desktop-only as a first step, we should also probably have build-flags for:

1. Allow to set up clusters dynamically;
2. Allow to persist user-created clusters.

Concerns

There may be little concerns in the desktop version as long as we implement the code correctly to avoid exploits. But the in-cluster deployments, if we even allow this, we should make sure that e.g. any user's sensitive data like authentication, always remains on the user side, i.e. tokens, client certs, need to be always passed by client.

If we implement this, I also wonder if we should stop loading the kube configs by default, and instead have a more elaborate "first-use" cluster setup for the desktop version.

cc/ @illume , @ashu8912 , @lijianzhi01

[joaquimrocha]

@ashu8912 has raised the point that maybe we don't need to give the option to persist dynamically created clusters. Thinking about it again, while I am not sure if it's something we need, I do agree that we can do without it for now and add the option later if needed, while the opposite is more difficult/compromising. Good point!