Kubernetes: pipeline jobs created should include labels

[max-rocket-internet]

Currently the jobs that are created to run pipelines are missing the standard labels. I think we should at least add the label k8s-app=drone for jobs. This is standard for almost every resource on k8s and is currently missing. This would allow a user to run kubectl delete job -l k8s-app=drone for example.

I think we should also consider adding more labels. e.g. for build ID, repo, etc. I see we have annotations for these but not labels. Are the annotations used/read by either Drone itself or drone/controller? If not, then I would move all the annotations to labels. If they are used, then duplicate some of this data to the labels so they can be used with kubectl <verb> -l xxx=yyy 🙂

[bradrydzewski]

Are the annotations used/read by either Drone itself or drone/controller

the only place we are really using labels is for configuring services:

kind: Service
metadata:
  name: mysql
spec:
  ports:
  - port: 3306
    targetPort: 3306
  selector:
    io.drone.step.name: mysql
---
kind: Pod
metadata:
  labels:
    io.drone.step.name: mysql

[max-rocket-internet]

the only place we are really using labels is for configuring services

Ah OK, well that's just a selector.

To be clear, here's what I mean, here's a description of a k8s job that was created by Drone to run a pipeline

$ kubectl describe job drone-job-8qxzn
Name:           drone-job-8qxzn
Namespace:      default
Selector:       controller-uid=35f09d9b-2ae9-11e9-8902-0a7cf1d29332
Labels:         controller-uid=35f09d9b-2ae9-11e9-8902-0a7cf1d29332
                job-name=drone-job-8qxzn
Annotations:    io.drone: true
                io.drone.build.id: 1
                io.drone.repo.id: 265
                io.drone.stage.arch: amd64
                io.drone.stage.created: 2019-02-07 15:01:12 +0000 UTC
                io.drone.stage.id: 1
                io.drone.stage.number: 1
                io.drone.stage.os: linux
                io.drone.stage.scheduled: 2019-02-07 15:01:13.202300802 +0000 UTC m=+329.412470876
Parallelism:    1

See all the annotations there? If they aren't used by drone itself, then I think all those key values should be moved to labels. And then it would also be good to have a label for the name of the repo. So perhaps it would look like this:

$ kubectl describe job drone-job-8qxzn
Name:           drone-job-8qxzn
Namespace:      default
Selector:       controller-uid=35f09d9b-2ae9-11e9-8902-0a7cf1d29332
Labels:         controller-uid=35f09d9b-2ae9-11e9-8902-0a7cf1d29332
                job-name=drone-job-8qxzn
                k8s-app=drone
                component=pipeline
                io.drone.repo: max-rocket-internet/drone-ci-test
                io.drone: true
                io.drone.build.id: 1
                io.drone.repo.id: 265
                io.drone.stage.arch: amd64
                io.drone.stage.created: 2019-02-07 15:01:12 +0000 UTC
                io.drone.stage.id: 1
                io.drone.stage.number: 1
                io.drone.stage.os: linux
                io.drone.stage.scheduled: 2019-02-07 15:01:13.202300802 +0000 UTC m=+329.412470876

The reason for this is that labels in k8s are very important. Labels are used in getting, viewing and deleting resources. Labels are using in logging infrastructure, for example in ELK, Splunk, Graylog, fluentd, stackdriver etc. Labels are using in monitoring tools, like Prometheus and Sysdig. etc etc.

Once you have a complex environment with multiple clusters, applications and environments then correct labels are essential for managing that complexity.

[tboerger]

Makes sense to me, if we don't need the annotations let's just move it to labels, otherwise duplicate them

[willejs]

I would be nice to also have a label with the job stage name and if its a deployment.

[bradrydzewski]

fixed by https://github.com/drone-runners/drone-runner-kube/commit/82dce67ad2652a17817df1b970e702ae86161135

new kubernetes runner documentation can be found here: https://kube-runner.docs.drone.io/