install tektoncd/pipeline in our test clusters

[jlewi]

It might be valuable to support tektoncd/pipeline for writing E2E tests.

One of the nice things about pipelines is that pipelines are split into reusable tasks. So we could have common tasks for things like deploying kubeflow and then combine these tasks into E2E tests.

Installing it should just be a matter of running kubectl apply in the test cluster. https://rubygems.pkg.github.com/tektoncd/pipeline/blob/master/docs/install.md

Related to: kubeflow/kubeflow#3035

[kkasravi]

@swiftdiaries here's my status

1. I'm working through this example.
2. It installs knative as well as TektonCD, one thing I was looking into is whether we can leverage tests and the test framework that kubernetes has so the pipeline would call serverless snippets similar to what kubetest does. To this extent who does this ticket and #425 isn't too important from my perspective but if we can leverage the 100s of e2e tests in kubernetes/test_infra it would be a huge time savings.

[jlewi]

@kkasravi Why would the E2E tests in kuberrnetes/test_infra be relevant to us? Wouldn't those tests be testing things we don't need to test?

[kkasravi]

@jlewi test_infra ends up calling ginkgo and uses kubernetes/test/e2e/framework which has lots of convenience functions for checking deployments, creating pods, etc

right now i'm just basically doing what kfctl_go_test does and evaluating whether this is a good approach and whether it's easy to mix python, golang in a tekton pipeline.

[nikhil-thomas]

@kkasravi You might find the following resources useful:

1. Tekton slack channels: tektoncd.slack.com
2. A simple build-deploy pipeline example: https://github.com/hrishin/voting-tekton-pipeline-samples (from a talk we delivered last Friday, the video is not published yet)
3. Tekton Pipelines Intro: https://www.youtube.com/watch?v=-ji5Z0qJmJs
   • source: https://github.com/nikhil-thomas/tekton-tutorial

If you have questions feel free to ask me directly or on Tekton Slack.

[kkasravi]

thanks @nikhil-thomas I've joined the slack channel.

[nikhil-thomas]

@kkasravi in i see that you have embedded all tekton resources in tektoncd/tektoncd/base. (in https://github.com/kubeflow/manifests/pull/264)

You might find the tektoncd/operator project very useful (https://github.com/tektoncd/operator). you will be able to setup tektoncd-pipelines components using manifests in this directory (https://github.com/tektoncd/operator/tree/master/deploy)

at present, the operator can install tektoncd-pipelines components (crds, rbac, deployments). it also gives the flexibility to install tektoncd-pipelines into a namespace other than tekton-pipelines.

you can reach us on operator channel on tektoncd.slack.com

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in one week if no further activity occurs. Thank you for your contributions.

[jlewi]

Still relevant to our efforts to begin using Tekton

[jlewi]

I think we want to use our release clusters not our test clusters.

The primary use case for this will be CD for our published artifacts. We don't want our CI clusters to have access to the registries where we host our images.

Our release infra is defined here: https://github.com/kubeflow/testing/tree/master/release-infra

Folks with access to the release infra are defined here: https://github.com/kubeflow/internal-acls/blob/master/release-team.members.txt