search

kubeshop / testkube

☸️ Kubernetes-native testing framework for test execution and orchestration
https://testkube.io
Other
1.37k stars 134 forks source link

Architecture: rebase testkube ontop of argo-workflows #4069

Open gberche-orange opened 1 year ago

gberche-orange commented 1 year ago

Is your feature request related to a problem? Please describe.

As a testkube user, In order to benefit from the features available in argo workflow that are missing in test kube, I need testkube to leverage argo workflow while keeping differentiating testkube features

Currently, testkube is implementing its features on top of plain k8s api, generating Job and CronJob resources, with init container fetching git sources and custom triggers.

Describe the solution you'd like

TestKube to leverage argo workflows for most of the facilities, and focus on missing and test-specific features instead

Additional context

Following are some thoughts on requirements in my team for a test facade, and how testkube could leverage argo workflows to support them.

Actors:

Key desired features of an off the shelf K8S test framework

features sub-feature Argo-workflow TestKube
K8S native authoring via CRD ~OK OK
Git-ops ready Mount content from Gitops repo To-test OK
Git-ops ready Trigger when changes to workflow/script content To-test (gh webhook) https://github.com/kubeshop/testkube/issues/4007
Git-ops ready Trigger on external conditions: SUT (system-under-test) changes To-test (gh webhook) https://github.com/kubeshop/testkube/issues/3712 https://github.com/kubeshop/testkube/issues/4017
Web ui ease of use, user friendly Weak Good
Web ui auth OK OK
Web ui trigger tests/scripts through custom input web forms ~ OK (basic enums) https://github.com/kubeshop/testkube/issues/4030
Web ui logs archiving OK OK
Web ui artefacts archiving OK OK
Web ui artefacts visualization To-test https://github.com/kubeshop/testkube/issues/4029
Web ui prometheus metrics To-test OK
Web ui notifications To-test To-test
Community size, maturity Large Small
Community support To-test OSS or enterprise/cloud subscription
Community reuseable templates To-test Builtin Test executors

argo workflow details

trigger when changes to script/workflow content

https://github.com/argoproj/argo-workflows/issues/8415 Conditionally Run Workflows Based on Git Artifact Files Changed

webui

Use cases

reuseable templates

https://github.com/argoproj-labs/argo-workflows-catalog/tree/master/templates few unmaintained templates

resulting test artifacts

https://argoproj.github.io/argo-workflows/artifact-visualization/

Prometheus metrics

https://argoproj.github.io/argo-workflows/metrics/

Trigger on git change

Likely through a webhook from the git repo provider (e.g. github or gitlab) https://argoproj.github.io/argo-workflows/events/#workflow-template-triggered-by-the-event https://argoproj.github.io/argo-workflows/webhooks/

The secret argo-workflows-webhook-clients tells Argo: What type of webhook the account can be used for, e.g. github. What "secret" that webhook is configured for, e.g. in your Github settings page.

https://argo-cd.readthedocs.io/en/stable/operator-manual/webhook/

Argo CD polls Git repositories every three minutes to detect changes to the manifests. To eliminate this delay from polling, the API server can be configured to receive webhook events. Argo CD supports Git webhook notifications from GitHub, GitLab, Bitbucket, Bitbucket Server and Gogs. The following explains how to configure a Git webhook for GitHub, but the same process should be applicable to other providers.

CRD schema

Argo workflow CRD schema are currently empty because the schema size exceeds the K8S supported size. See https://github.com/argoproj/argo-workflows/issues/8190 and https://github.com/argoproj/argo-helm/issues/2105

Mount content from gitops repo

apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
  name: osb-smoke-test-template
spec:
  entrypoint: osb-smoke-test
  arguments:
    artifacts:
      - name: myrepo
        git:
          repo: https://
          branch: main
          depth: 1
          disableSubmodules: true
          usernameSecret:
            name: name
            key: key
          passwordSecret:
            name: name
            key: key

Related issues:

gberche-orange commented 1 year ago

thanks to @dejanzele and @TheBrunoLopes for our 2nd sync up meeting today on this topic and your sharing of current testkube dogfooding:

TestKube helm release is tested using testkube, covering test kube api using postman executor, and playright executor (+ executor smoke tests)

vsukhin commented 1 year ago

4017 is done

4007 is in progress

senare commented 1 year ago

Looks promising !

I got 2 questions (or suggestions maybe?)

First when it comes to auth and authn ... would it be possible to stand on the shoulders of ArgoCD here ? I just configured ArgoCD <= SAML => Authentik and then added ArgoWorkflow <= OIDC => DEX => ArgoCD (I think thats the flow atleast)

Then we also have Argo Events, which I was looking to add in to the mix for triggering the right flow and coordination etc ... but when you describe it above it is not mentioned, would you care to elaborate as to why a bit ?

LINKS