Kubebenchjob API v1alpha2

[xyhuang]

The following changes are made in v1alpha2 design:

• Remove secretSpecs field, the secret configs can be handled by user directly by mounting to workflow agent or workload resources through the API.
• Add workflowAgent field, that lets user explicitly configure the workflow agent specifications. This can be backed by a default config that is fed into the operator separately, so that if the field is omitted by the user the default values will be taken.
• Turn volume configs into 2 fields: managedVolumes and volumes, where managedVolumes are the special volumes shared between workflow agents and user's workload resources and directory structure managed by the workflow, and volumes are the general volumes that can be mounted by user to workflow agents and workload resources.
• Replace the "pre-job", "main-job", "post-job" concepts with DAG styled "tasks". The "tasks" look similar as Argo's DAG tasks, but come with a set of customized behaviors, such like autoWatch, etc. as well as special manifest references for arbitrary resources.
• Removereports field.

Example yaml manifest:

apiVersion: kubeflow.org/v1alpha2
kind: KubebenchJob
metadata:
  name: kubebench-job
  namespace: default
spec:
  serviceAccount: default
  volumes:
  - name: my-config-volume
    configMap:
      name: my-config-pvc
  - name: my-data-volume
    persistentVolumeClaim:
      claimName: my-data-pvc
  managedVolumes:
    experimentVolume:
      name: my-experiment-volume
      persistentVolumeClaim:
        claimName: my-experiment-pvc
    workflowVolume:
      name: my-workflow-volume
      persistentVolumeClaim:
        claimName: my-workflow-pvc
  workflowAgent:
    container:
      volumeMounts:
      - name: my-config-volume
        mountPath: /kubebench/config
  tasks:
  - name: pre-job-1
    container:
      name: my-prejob-1
      image: myprejob1-image:latest
  - name: pre-job-2
    container:
      name: my-prejob-2
      image: myprejob2-image:latest
  - name: main-job
    dependencies: [pre-job-1, pre-job-2]
    resource:
      manifestFrom:
        path: my/tfjob/parameter.yaml
      volumeMounts:
      - name: my-data-volume
        mountPath: /data
      options:
        mountManagedVolumes: true
        autoWatch:
          timeout: 5m
        autoDelete: true
        numCopies: 1
  - name: post-job
    dependencies: [main-job]
    container:
      name: my-postjob
      image: mypostjob-image:latest

[xyhuang]

/assign