deis / controller

Deis Workflow Controller (API)
https://deis.com
MIT License
41 stars 53 forks source link

Adding lifecycle hooks to deis-controller api #1311

Open Cryptophobia opened 7 years ago

Cryptophobia commented 7 years ago

Adding the lifecycle hooks feature to the deis-controller api. This allows users to attach lifecycle postStart and preStop hook commands to kubernetes api via the deis api.

Example of a preStop hook that would gracefully shutdown Resque worker processes: https://gist.github.com/Cryptophobia/447ecea7858d26141359b22161e70ee2

Example of the POST to deis.$hostname/v2/apps/testing-again/config:

JSON Body:

 "lifecycle_pre_stop": {
        "worker": "echo 'Running the postStop script' && chmod +x bin/kube-resque-pre-stop.sh 
&& ./bin/kube-resque-pre-stop.sh;"
    }

For long running preStop hooks the terminationGracePeriodSeconds needs to be increased for the worker pods:

deis config:set KUBERNETES_POD_TERMINATION_GRACE_PERIOD_SECONDS=900 -a testing-worker

The tests are not written yet. Code in deis-cli has not been written yet.

rvadim commented 7 years ago

This MR probably fixes #1254

Cryptophobia commented 7 years ago

@rvadim : Yes, it would give you a graceful showdown option now. However, you still need to make sure that bash shell is present in the docker container that you are deploying. I was originally testing with one of the docker example Go apps and interestingly, the bash shell is not in there...

Cryptophobia commented 6 years ago

@mboersma : How do I get passed these three workflow-e2e failures in the build, they do not seem to be related to the deis-controller changes here?

codecov-io commented 6 years ago

Codecov Report

Merging #1311 into master will not change coverage. The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1311   +/-   ##
=======================================
  Coverage   86.84%   86.84%           
=======================================
  Files          45       45           
  Lines        3944     3944           
  Branches      682      682           
=======================================
  Hits         3425     3425           
  Misses        348      348           
  Partials      171      171
Cryptophobia commented 6 years ago

The remote upstream/master was merged on top of the branch rather than rebase. Is that okay?

mboersma commented 6 years ago

Is that okay?

The project has preferred git rebase just to keep merge commits at a minimum, but no worries--not a blocker.

Looks like Jenkins failed this randomly. I'll run it again and 🤞.

Cryptophobia commented 6 years ago

@mboersma : Looks like the build is still stuck in pending? It looks like it passed when you view the text output log.

Cryptophobia commented 6 years ago

How do I get this one passed as well? Looks like it failed because of a network issue related to ?missing namespace? in the Kubernetes cluster?

mboersma commented 6 years ago

Jenkins, test this please.

Cryptophobia commented 6 years ago

Jenkins, will you please pass this? Jenkins, no don't eat that off the floor! Jenkins, sit! Jenkins, be nice to me and I'll give you a cookie. 🍪 🐩

Looks like it failed because the Jenkins server could not create the "deis" user group. Seems like a server error. @mboersma , what do you think?

adduser: `/usr/sbin/groupadd -g 106 deis' returned error code 10. Exiting.
Cryptophobia commented 6 years ago

This PR has been recreated for the https://github.com/deisthree fork here: deisthree/controller#3