driver: knative / tekton

[bradrydzewski]

I want to explore a knative (or tekton) runtime for Drone as a possible replacement for our experimental kubernetes runtime. The goal would be to have a yaml definition that is Kubernetes-first, and does not try to force-fit existing Drone concepts into Kubernetes.

kind: pipeline
type: knative

steps:
- name: hello
  image: busybox
  args: ["echo", "hello", "build"]

The above yaml would pretty much directly translate to a knative yaml

-kind: pipeline
-type: knative
+apiVersion: build.knative.dev/v1alpha1
+kind: Build
+metadata:
+  name: hello-build

spec:
  steps:
  - name: hello
    image: busybox
    args: ["echo", "hello", "build"]

This will better align Drone with the Kubernetes community and Kubernetes conventions. Any changes to the runtime would be handled by the Knative and Tekton working groups. Drone would focus its efforts toward higher level components and adding value on top of Knative and Tekton, including the font-end user experience, integrations, plugins, etc.

[zetaab]

the problem that I see in knative is that it requires istio AFAIK. Istio is not very common in kubernetes setups (yet)

[bradrydzewski]

It looks like they are actually moving the Build resource to Tekton, so Tekton would be responsible for the build and Knative is responsible for running. So it may be more accurate to say that Drone would support Tekton. I do not think Tekton requires Istio.

[zetaab]

not sure does that help at all if I compare to situation like now https://github.com/tektoncd/pipeline/blob/master/docs/install.md#configuring-tekton-pipelines

How are resources shared between tasks
Pipelines need a way to share resources between tasks. The alternatives are a Persistent volume or a GCS storage bucket

or at least it does not help in case of volume sharing

[bradrydzewski]

The goal with adopting Knative/Tekton is not to solve the volume problem per-se, the goal is to leverage a solution that is backed by a large community of Kubernetes experts. I believe these technologies will evolve quickly and improve rapidly; these projects have significant resources at their disposal.

Knative/Tekton are meant to be building blocks, which means these projects can handle the low level details (volumes, networking, namespacing, security), and Drone can handle the higher level details (user interfaces, permissions, plugins, version control integrations). This allows the core Drone team to stay a bit more focused.

Because Knative/Tekton are building blocks and do not have high level integrations, user interfaces, etc I think this is a also great opportunity for Drone to fill a gap and really add value to the Kubernetes community.

[zetaab]

In my opinion this sounds really good

[MOZGIII]

What I don't like tekton and knative is that they're a bit resource hungry out of the box

[tboerger]

If you dislike the resource usage you don't have to use it. There are different options.

[MOZGIII]

You mean there're other drone drivers for kubernetes?

[tboerger]

The currently existing one is experimental, but with the new runner-go framework anybody could build a proper native kubernetes runtime.

[MOZGIII]

I see. Well, I'm not very happy about it. "Build your own thing" is always an option, right? This is now why I'm participating in discussions at Drone CI... Excuse me if something's wrong.

[tboerger]

The home for the new upcoming runners is at https://github.com/drone-runners, I guess even the current experimental Kubernetes runtime will be moved there at some point.

[MOZGIII]

This looks good! Is there any issue dedicated to discussion of the new design? Or docs/blog?

[tboerger]

Not yet, Brad is currently working on the first 2 runners to demo the runner-go framework.

[zetaab]

@bradrydzewski any news on this?

[tboerger]

If there would be any news it would be written to this issue. Asking for an update doesn't speed it up. With the new runner-go library it should be quite simple to create custom runtimes. You can already see examples how this could look like within the drone-runners org, if you build such a runtime I think Brad would welcome it to move it into this organization.

[bradrydzewski]

@zetaab I am currently working on porting the Docker runner to the new runner-go library here. I expect this task will be done by end of week.

Once complete, the next step is to start working on the next version of the Kubernetes runtime. We have already started discussing design, but will not start build until the Docker runner is ported and we are freed up. Work could start as soon as next week.

We may start with a simple Kubernetes runner that tries to improve upon the previous iteration, specifically by running all pipeline containers in the same pod (as opposed to per-pipeline namespace) and using the Downward API to control execution (similar to Tekton). Or we may try and use Tekton directly. To be determined ...

[zetaab]

Great. I am waiting for tekton :)

[withlin]

I am waiting for tekton too.