kubernetes definitions directory

[bacongobbler]

In kubernetes there is a way to define all of the required RCs and services in one yaml/json file (as you mentioned in https://github.com/technosophos/prototype-repo/tree/master/def). Perhaps it would be better to leave kubernetes definitions in the root of the project as its own file? Perhaps something like kube.json?

I don't think there's any conventional standard set quite yet in the kubernetes world, so we kinda have the opportunity to create best practices here :)

[technosophos]

Very good point.

I was trying to work through some of the developer workflows to see how I do things, and I'd love to hear your input on your workflow. But I noticed that I do things like this:

• start up the service, add a secret volume, and start the rc
• do some testing, then fix some stuff in code
• stop the rc
• modify the rc definition
• restart the rc

In that case, I usually want the RC in a separate file so I can work with it in isolation.

Another use case, based on etcd, requires me to start all the services, then the discovery rc, then wait for a few seconds, then start the other rc. That's probably bad design on my part... but I'm not sure it will always be avoidable.

But at the same time, I really like the idea of having just one kube.json or kube.yaml file, and have it in the root...

As I think about it, I'm also wondering about whether we should talk about templating Kubernetes definitions. Certain fields like labels, annotations, image, and replicas change pretty frequently during development. I wonder if we should do something like store a template version of the file (maybe just using Go templates).

You're right about best practices for sure. Not much has been done...

[bacongobbler]

As I think about it, I'm also wondering about whether we should talk about templating Kubernetes definitions. Certain fields like labels, annotations, image, and replicas change pretty frequently during development. I wonder if we should do something like store a template version of the file (maybe just using Go templates).

That may be something we have to strongly consider, and could change how we structure it back to the def convention. Maybe we could write a tool that would template out the Kubernetes templates for us? Something like python setup.py could work for us.

e.g.

$ vim ~/.<name>/config # define remote registry and kubernetes cluster
$ <name> build # identical to `make build push` in deis/deis, just builds the Docker image
$ <name> deploy # kubectl create, pointing to the built image and the external kubernetes cluster

[technosophos]

Hmmm... that is a really good idea. I imagine that a combination of a file like setup.py and Go's built-in template engine could make a lot of this very easy.

Essentially, our new build/release process has two build artifacts:

• the Docker image
• the Kubernetes definition file(s)

A little bit of build process around those would be great. And the kind of cool thing is that our final releases for many of these projects may be nothing other than the kube definitions (assuming the images are uploaded to a public Docker registry).

[technosophos]

Per @bacongobbler 's offline suggestion, I changed the def/ directory to manifests/, which is what Kubernetes uses.

[bacongobbler]

closing in favour of helm.