Kubernetes support

[eldada]

I have a use case where I need this exact functionality, but I want to implement it in Kubernetes. Start the VM container as a pod (managed with a StatefulSet).

This will allow for easy scale up and down and better use of common resources.

Do you have an example of such an implementation?

[Skarlso]

That is an interesting idea. But I'm not sure I fully understand your use case. Would you mind elaborating?

Kubernetes is the one managing the Pods life cycle. How would footloose fit into this?

[eldada]

Well, my use case is to use ssh connection to a VM. So the UX is VM oriented and I don't need to mess around with kubectl setup that might be complex and cumbersome. Examples:

• I want to test a script on a special Linux flavour
• I need to test installers (RPM/Deb)

In general - it's VMs as a service like any other managed hypervisor, but the backend is running in K8s, which means provisioned VMs are actually pods. I think this will improve cost and speed.

Looks like you have the core ready - the Docker container. Now need to provision it in K8s. I admit I didn't dig in to the implementation details...

[Skarlso]

Yeah, for that to work footloose would have to implement the K8s API including auth and all to provision on the k8s cluster. I don't think that will happen. :D It's rather cumbersome. Not to mention that would have to eventually deal with all the problems of the k8s clustering and scheduling. Where the containers are, why, labels and so on and so forth.

[dlespiau]

Oh that's an interesting idea! I missed the issue, sorry.

This seems completely doable. The core of footloose is really the set of "VM images" eg. quay.io/footloose/centos7. It should be somewhat straightforward to start a pod with this image and start /sbin/init.

There are (at least) a couple of ways to think about it:

1. We now have two "backends": docker and a ignite. We could add a kubernetes backend and things would work through the footloose CLI.

2. Re-implement footloose with Kubernetes objects and "just" use those through kubectl. It gives you the full access to the Kubernetes API.

The main problem I see is what should be done to expose the SSH services to the outside world. Maybe port-forward the ssh port through the API server (a la kubectl port-forward), expose each service with an ingress controller, install an ssh proxy in the cluster and only expose this service, ...

[Skarlso]

But that would mean that footloose would mess with the orchestration of Pods, not?

[dlespiau]

for 1., the way I see it is that footloose would create a pod (maybe a static pod, maybe through a statefulset) by talking to the Kubernetes API server and run a footloose container image there. Kubernetes is still responsible for orchestrating the pod somewhere on the cluster.

In that scenario, Kubernetes would replace the docker daemon in what we currently have, effectively enabling managing a fleet of footloose containers on the Kubernetes nodes.

[Skarlso]

Yes, like I said, it would have to implement talking to the kube api which means auth and certificate handling and such. Which might not be a big problem, just saying. :)

[dlespiau]

Ah yes, of course, we'd need to authenticate :) the Kubernetes client libraries do that for you, parse kubeconfig files and it's really a few lines of code to create a Kubernetes client: https://github.com/ericchiang/k8s#creating-out-of-cluster-clients

[stealthybox]

some options are:

• shell out to kubectl, kubectl apply/kubectl wait/kubectl delete (super simple)
• vendor kubectl (difficult?)
• use client-go with versioned API's for a small number of Kinds (super type-safe)

I think using kubectl port-forward for ssh sounds fine.