proposal: pipeline as a first-class kabanero citizen

[gireeshpunathil]

At present, the pipeline is an instance of tekton pipelines customized for kabanero projects. Few pain points with this approach:

• there is a steep learning curve involved with anyone who wants to make use of a pipeline
• for an enterprise user, there are several modifications required at the moment, plus many kubectl / oc tasks as pre-reqs
• the pipeline in its current form do not conceal any yaml-complexities and k8s-internals

The proposal is to integrate pipeline as a first-class citizen into kabanero capability by providing a high level abstraction on its ingredients: more specifically, the tasks and resources. The objective is to expose an interface through which a user can define their own pipeline with the help of interactive with an intuitive workflow model.

Proposal 1:

Define a custom grammer, specification and a template for defining pipelines that the user can customize. It still sues YAML syntax, but the verbs and qualifiers are made more simpler, and high level.

Expose a command line interface that will compile the high level pipeline description manifest, and generate a fully-blown working set of pipelines that can be applied to k8s clusters.

Proposal 2:

Expose a command line interface that interacts with the user and constructs a pipeline from scratch, through a series of questions. Much more flexible than first one, but much more complex to implement.

these can be a fit for upstream tekton/pipeline project as well, but the advantage of kabanero are:

• this capability is scopped well within kabanero's stated purpose
• it already has a cmd

LMK WDYT!

[stephenkinder]

@gireeshpunathil I'm wondering if perhaps there is a general need to address this issue with Tekton itself? I agree that if Kabanero introduced another orchestration engine as an opinionated alternative, that it may indeed be nice to abstract the "pipeline" or "workflow" template that Kabanero would translate. It might be reasonable to improve Tekton and extend Kabanero.

[gireeshpunathil]

I have been thinking about ways to represent tasks and resources and ways to correlate them in an intuitive way, while not introducing a totally new semantics. Looking at the jenkins' declarative pipeline syntax, I see the verbosity is less, but guess we could do even better:

Here is what I come up with:

Re-use the Dockerfile semantics for pipeline workflow composition, with most of the verbs will carry the same meaning, while reducing the declarations drastically.

For example, a targz pipeline can be abstracted thus:

LABEL targz
FROM ubuntu
ARG input=http://example.com/archive.tar.gz
ENV foo=bar
RUN tar xzvf $input
RUN cat source/file.txt