sky-uk / kfp-operator

https://sky-uk.github.io/kfp-operator
BSD 3-Clause "New" or "Revised" License
15 stars 2 forks source link

Move provider annotation to spec of resource #345

Closed grahamia closed 5 days ago

grahamia commented 3 months ago

Currently it is an annotation, but the requested provider can be moved to be a mandatory field in the spec of all the KFP-Operator custom resources.

Before this card:

apiVersion: pipelines.kubeflow.org/v1alpha5
kind: Pipeline
metadata:
  name: penguin-pipeline
  annotations:
    pipelines.kubeflow.org/provider: kfp
spec:
  image: kfp-quickstart:v1
  tfxComponents: base_pipeline.create_components
  env:
  - name: TRAINING_RUNS
    value: 100

After this card:

apiVersion: pipelines.kubeflow.org/v1alpha6
kind: Pipeline
metadata:
  name: penguin-pipeline
spec:
  provider: kfp
  image: kfp-quickstart:v1
  tfxComponents: base_pipeline.create_components
  env:
  - name: TRAINING_RUNS
    value: 100

Note that it will be in Spec.Run.Provider in RunConfigurations, rather than just in Spec.Provider, as the RunConfiguration references a Run:

spec:
  run:
    provider: kfp
    pipeline: some-pipeline
  triggers:
    onChange:
    - pipeline
    schedules:
    - 0 0 12 * * *

This then enforces that a provider must be specified for the resource.

Add this change to the v1alpha6 version

The default provider can't be removed from the KFP Operator configuration till v1alpha7 is created and we guarantee that provider is always populated (i.e. we drop alpha5 support)