search

molgenis / molgenis-ops-helm

Operations Helm charts to deploy build infrastructure on Kubernetes
GNU Lesser General Public License v3.0
2 stars 13 forks source link

readme

Build Status

MOLGENIS - Helm templates

These are the Helm templates that we will use for MOLGENIS operations. Basic concepts in respect to docker you need to know.

Deployments

Are a set of pods that will be deployed according to configuration that is usually managed bij Helm. These pods interact with eachother by being in the same namespace created by kubernetes according to the deployment configuration.

Pods

A pod is wrapper around a container. It will recreate the container when it is shutdown for some reason and interact with other pods when needed.

Containers

A container is a docker-container that is created from a docker image. It could be seen as an VM for example

Images

An image is a template for a container some sort of boot script but also contains the os for example. A build dockerfile, if you will.

Prerequisites

There are some prerequisites you need.

Kubernetes

When you want to use kubernetes there are some commands you need to know. Also running on a remote cluster will be a must have to control your whole DTAP.

Useful commands

Commands that can be used to get information from a kubernetes cluster

Pods

Services

Volumes

Deployments

Remote clusters

When you want to see what is running on the clusters at the CIT you have to make a context switch. You can access the cluster with kubeconfig-files. You can obtain these by downloading them from the MOLGENIS kubernetes cluster.

or when you placed the MOLGENIS configuration besides the original one

kubectl config use-context molgenis --kubeconfig=full path to molgenis config

- You can now access all facilities of the MOLGENIS cluster like it is running locally

  *Example:*
```bash
kubectl get pods --namespace=*#namespace of application#*

Helm

This repository is serves also as a catalogue for Rancher. We have serveral apps that are served through this repoistory. e.g.

Useful commands

You can you need to know to easily develop and deploy helm-charts

Chart testing

Jenkins will test the chart for you, but to test it locally you can use the docker image, see ct.sh

Persistence

The manage your pv's you have to make a distinction between retainable pv's and non-retainable pv's.

The status "released" is the keyword that the volume is not attached to a deployment anymore.

Cleanup old pv's

Fetch all released pv's to check if they are all released.

kubectl get pv | grep Released

Then remove them permanently.

kubectl get pv | grep Released | grep -o '^\S*' | grep . | xargs kubectl delete pv

Orphaned kubernetes resources

You can terminate orphaned resources can be a pain. We described how to deal with 2 of them.

Pods

Sometimes pods won't die on themselves and you need to help them a little.

kubectl remove pod #pod name# --namespace=#namespace# (optional: [--force] [--grace-period=0])

Removes a pod from the system (but will restart if the option is set in the deployment,yaml [see note]).

note: You can not do this while the deployment of the service is still there

Namespaces

To permanently terminate the namespace you have to catch the JSON output in a file.

kubectl get namespace molgenis-sentry -o=json > sentry.json

Then you need delete some parts of the namespace JSON to purge the repo.

{
    "apiVersion": "v1",
    "kind": "Namespace",
    "metadata": {
        "annotations": {
            "cattle.io/appIds": "molgenis-sentry",
            "cattle.io/status": "{\"Conditions\":[{\"Type\":\"InitialRolesPopulated\",\"Status\":\"True\",\"Message\":\"\",\"LastUpdateTime\":\"2018-12-17T15:08:28Z\"},{\"Type\":\"ResourceQuotaInit\",\"St
            "field.cattle.io/creatorId": "u-6nb8b",
            "field.cattle.io/projectId": "c-rrz2w:p-fsjx8",
            "lifecycle.cattle.io/create.namespace-auth": "true"
        },
        "creationTimestamp": "2018-12-17T15:08:58Z",
        "finalizers": [
            "controller.cattle.io/namespace-auth"
        ],
        "labels": {
            "cattle.io/creator": "norman",
            "field.cattle.io/projectId": "p-fsjx8"
        },
        "name": "molgenis-sentry",

        // START DELETE
        "resourceVersion": "21694313",
        // END DELETE

        "selfLink": "/api/v1/namespaces/molgenis-sentry",
        "uid": "add523b7-020d-11e9-ac6d-005056b29ae4"
    },

    // START DELETE 
    "spec": {
        "finalizers": [
            "kubernetes"
        ]
    },
    // END DELETE

    "status": {
        "phase": "Active"
    }
}

Then when you determined the cluster name with rancher cluster you can enter it where #cluster# stands and you can fill the target namespace where #target-namespace# stands.

Before you can access the kubernetes API, you need to make it locally available by executing the following command:

rancher kubectl proxy --port=8001 &

Then execute the curl.

curl -k -H "Content-Type: application/json" -X PUT --data-binary @sentry.json http://127.0.0.1:8001/k8s/clusters/#cluster#/api/v1/namespaces/#target-namespace#/finalize

This should be the result:

{
  "kind": "Namespace",
  "apiVersion": "v1",
  "metadata": {
    "name": "molgenis-sentry",
    "selfLink": "/api/v1/namespaces/molgenis-sentry/finalize",
    "uid": "e48cb533-01dd-11e9-ac6d-005056b29ae4",
    "resourceVersion": "21692263",
    "creationTimestamp": "2018-12-17T09:26:54Z",
    "deletionTimestamp": "2018-12-17T09:50:01Z",
    "labels": {
      "cattle.io/creator": "norman",
      "field.cattle.io/projectId": "p-fsjx8"
    },
    "annotations": {
      "cattle.io/status": "{\"Conditions\":[{\"Type\":\"InitialRolesPopulated\",\"Status\":\"True\",\"Message\":\"\",\"LastUpdateTime\":\"2018-12-17T09:26:24Z\"},{\"Type\":\"ResourceQuotaInit\",\"Status\":\"True\",\"Message\":\"\",\"LastUpdateTime\":\"2018-12-17T09:26:23Z\"}]}",
      "field.cattle.io/creatorId": "u-6nb8b",
      "field.cattle.io/projectId": "c-rrz2w:p-fsjx8",
      "lifecycle.cattle.io/create.namespace-auth": "true"
    }
  },
  "spec": {

  },
  "status": {
    "phase": "Terminating"
  }
}%