search

tintoy / dotnet-kube-client

A Kubernetes API client for .NET Standard / .NET Core
MIT License
190 stars 32 forks source link

Coding Rollout Pause/Resume/Rollback #60

Open brobichaud opened 5 years ago

brobichaud commented 5 years ago

It's unclear to me how I would go about acting on a deployment rollout. I'm trying to figure out how to Pause, Resume and Rollback a rollout.

Initially I thought I could Pause/Resume by patching the deployment.Spec.Paused field as seen below. But this does not work.

var depClient = GetClient().DeploymentsV1();
var dep = await depClient.Get(deploymentName);
await depClient.Update(dep.Metadata.Name, patch =>
{
   patch.Replace(d => d.Spec.Paused, true);
});

I also have no idea how I would Rollback. Any ideas on this area of the api?

tintoy commented 5 years ago

Hi - am overseas at the moment with patchy internet and no laptop, can you give me a couple of days? Will have a look at this as soon as I’m able.

brobichaud commented 5 years ago

Yeah of course. I totally appreciate your help in any way and I get that this is maybe more about how the k8s api works. So thank you for taking a look when you get a chance!

Loving your nuget btw! :-)

tintoy commented 5 years ago

Hi - I'm now back in front of my computer :)

Generally, I run kubectl <command> --v=8 (note the double-dash) to find out what API calls are required to implement a kubectl command. At that verbosity level, it will show you all the K8s API calls it is making as part of that command.

tintoy commented 5 years ago

See here for more information about kubectl verbosity.

If I get a chance this weekend, I'll have a look at this output myself and see if I can work up a sample (although if you work it out before I do, I'm never too proud to take a PR ;-D ).

brobichaud commented 5 years ago

Whoa, I was unaware of the --v option. Nice! In playing with it for a few minutes, does this make it easy for you to map in your head the k8s api call to objects in your library? It was not obvious to me how that mapping works.

I will take a look as soon as i can, though my attention has been redirected at other areas for a bit. I do need to get back to this as soon as I can. Totally appreciate the pointer to this option! Oh, and thanks for continuing to build a nice abstraction layer over the api, sooo appreciated!

tintoy commented 5 years ago

For pause and resume, just update the deployment setting paused to true. For rollback, not sure yet.

tintoy commented 5 years ago

Sorry, was sick over the weekend so I haven't had time to look too deeply into this yet.

My instincts tell me the rollback stuff relates to linking of a Deployment resource and one or more associated ReplicaSet resources via labels / annotations (each ReplicaSet representing an "entry" in the Deployment's history). I'll try to find some time this week to actually have a go at running kubectl to see what resources it touches to perform this command.

tintoy commented 5 years ago

Yep, looks like I was right (in general, at least):

https://unofficial-kubernetes.readthedocs.io/en/latest/concepts/workloads/controllers/deployment/#revision-history-limit

brobichaud commented 5 years ago

Yep, looks like I was right (in general, at least):

https://unofficial-kubernetes.readthedocs.io/en/latest/concepts/workloads/controllers/deployment/#revision-history-limit

Ah, yeah that does indicate the Paused field should work. I played with this a bit back before I added this issue and was not having luck. I will take another look at it though. It does seem the obvious choice! And the rollbackTo I didn't even see.

tintoy commented 5 years ago

CC: @felixfbecker - might be useful for PSKubectl?

felixfbecker commented 5 years ago

Yup! Although I generally find that you can do everything with kubectl apply and source-controlled YAML files :)

tintoy commented 5 years ago

Ok, took some fiddling but I figured it out 🎉

See https://github.com/tintoy/dotnet-kube-client/blob/develop/samples/DeploymentWithRollback/Program.cs for details :)

brobichaud commented 5 years ago

I can verify that pause/resume works as you indicated. Thank you. Looks like you're working on some enhancements to make rollback better supported, I'll wait on those.

tintoy commented 5 years ago

Published v2.2.6.

brobichaud commented 5 years ago

I am finally getting back to this. It is fundamentally working but I am seeing a couple of issues.

The first was with the revision annotation. I had to reverse the order of the patch and update the annotations first then update the template spec. After that the annotations look correct.

But I am also seeing that the labels in the new pods are not being set correctly. If I deploy rev 1 with a label of app=1, the deploy rev 2 with a label of app=2. Then rollback I am seeing the pods have a label of app=2, when it should be app=1.

Don't the pods get their labels from the replicaset as they are created? Or maybe the deployment? Do I need to update the labels of the deployment?

Any thoughts on this?

tintoy commented 5 years ago

Hmm, when running the same operation via kubectl (with —v=10), is it sending the same requests?

I must admit you’re going deeper into the annotations jungle than I’ve dared to venture so far :)

tintoy commented 5 years ago

I believe the pods get their labels from the replica set’s pod-template section of its spec (not the labels of the replica set itself) though.

tintoy commented 5 years ago

(spec.template.metadata.labels, not metadata.labels)

tintoy commented 5 years ago

If you’re working with a Deployment, then it’s still the same deal I think.