sky-uk / kfp-operator

https://sky-uk.github.io/kfp-operator
BSD 3-Clause "New" or "Revised" License
15 stars 2 forks source link

Add start and end datetime to schedules #364

Closed grahamia closed 1 week ago

grahamia commented 2 months ago

Is your feature request related to a problem? Please describe. To save on cost and the inconvenience of having to arrange deployments etc at the right time (to start and end a training schedule) it would be nice to be able to specify the start and end time for a schedule to run.

Describe the solution you'd like In the runconfiguration resource as part of the schedule trigger you should be able to optionally also set a start and end date, therefore also transferred into the runschedule resource the same information. Given this would need a new version of the resources as string will need to a struct containing the crontab and optional start and end datetimes.

These should be optional and if not given then as it is now, the schedule starts on deployment and doesn't end until removed.

RunConfiguration CR before:

apiVersion: pipelines.kubeflow.org/v1alpha5
kind: RunConfiguration
metadata:
  name: penguin-pipeline-recurring-run
spec:
  run:
    pipeline: penguin-pipeline:v1-abcdef
    ...
  triggers:
    schedules:
    - '0 * * * *'
    onChange:
    - pipeline
    runConfigurations:
    - dependency-rc

RunConfiguration CR after:

apiVersion: pipelines.kubeflow.org/v1alpha6
kind: RunConfiguration
metadata:
  name: penguin-pipeline-recurring-run
spec:
  run:
    pipeline: penguin-pipeline:v1-abcdef
    ...
  triggers:
    schedules:
    - cronExpression: '0 * * * *'
      startTime: "2024-09-24T17:24:43Z"
      endTime: "2024-10-24T17:24:43Z"
    onChange:
    - pipeline
    runConfigurations:
    - dependency-rc

RunSchedule CR before

apiVersion: pipelines.kubeflow.org/v1alpha5
kind: RunSchedule
metadata:
name: penguin-pipeline-abcde
spec:
  artifacts:
  - name: pushed_model
    path: Pusher:pushed_model:0[pushed == 1]
  pipeline: penguin-pipeline
  schedule: 0 0/3 * * *

RunSchedule CR after

apiVersion: pipelines.kubeflow.org/v1alpha6
kind: RunSchedule
metadata:
name: penguin-pipeline-abcde
spec:
  artifacts:
  - name: pushed_model
    path: Pusher:pushed_model:0[pushed == 1]
  pipeline: penguin-pipeline
  schedule: 
    cronExpression: 0 0/3 * * *
    startTime: "2024-09-24T17:24:43Z"
    endTime: "2024-10-24T17:24:43Z"

Additional context Add any other context or screenshots about the feature request here.

Vertex AI provider accepts start and end datetime as parameters to its createSchedule api

Kubeflow also allows it through using APICronSchedule which we currently use and has StartTime and EndTime attributes.

Tasks