search

kubeflow / spark-operator

Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.
Apache License 2.0
2.75k stars 1.36k forks source link

Trigger the next scheduled spark application ahead of time #1606

Open joseparajelesGL opened 2 years ago

joseparajelesGL commented 2 years ago

Currently the ScheduledSparkApplication creates a next run on the next window of the cron configuration, could it be possible to have an annotation that tells the controller to trigger a new run? This will be particularly useful when testing application with long cron schedules.

yliu224 commented 1 year ago

@joseparajelesGL Did you find any temporary solution for this one?

joseparajelesGL commented 1 year ago

@joseparajelesGL Did you find any temporary solution for this one?

@yliu224 I had to do a patch to the status subresource. You will need an updated version of kubectl (1.24+) or a client that can update status

I used this patch

kubectl patch --subresource status -n $NAMESPACE  ScheduledSparkApplication $SCHEDULED_SPARK_APPLICATION --type json -p '[{"op":"replace", "path":"/status/nextRun", "value":"2023-01-01T00:00:00Z"}]'

That will make the system run it next. If you want an specific time feel free to change it to the proper time

github-actions[bot] commented 2 weeks ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.