spring-cloud / spring-cloud-dataflow

A microservices-based Streaming and Batch data processing in Cloud Foundry and Kubernetes
https://dataflow.spring.io
Apache License 2.0
1.12k stars 583 forks source link

Allow modification of existing schedule #3810

Open RafiGreenberg opened 4 years ago

RafiGreenberg commented 4 years ago

Problem description: Currently (2.4.0-RELEASE) once you have scheduled a task, there is no way to modify the schedule or it's arguments/parameters. You configure the name, cron expression, arguments, and parameters when you first create the schedule, and then your only option is to delete and recreate it if it needs to be changed. Also there is no way of seeing which arguments/parameters were configured. You can only see the cron expression on the schedule details page.

Solution description: Allow modifications to the cron expression, arguments, and parameters for a scheduled task. Also display all these settings in the schedule details page.

ntantri commented 3 years ago

@sabbyanandan I would like to start exploring this ticket. Hope this task is still relevant?

sabbyanandan commented 3 years ago

Hi, @tan31989. Yes, very much so! Please when you go about implementing it, you will have to consider both CF and K8s backend hooks, so on platforms where pausing is possible, we could enable it.

ntantri commented 3 years ago

Hey Folks,

While looking into the code, I saw the schedules are being setup using this service class SchedulerService.java , which for now has a couple of methods only around schedule, unschedule and unscheduleForTaskDefinition.

This eventually means, we could unschedule a given modification of a schedule and re-schedule using the schedule

The solution talks about:

  1. Allow modifications to the cron expression, arguments, and parameters for a scheduled task.
  2. Also, display all these settings on the schedule details page.

For this wanted to quickly dig to get your thoughts on this:

For 1. (Allow modifications to the cron expression, arguments, and parameters for a scheduled task) - can we look forward to unschedule the created task and schedule a new one?

@ilayaperumalg mentioned @cppwfs who wrote the initial spec. Correct me if I am wrong.

cppwfs commented 3 years ago

Hello @tan31989,

Thank you for question!

A schedule update method for SCDF it would require the following steps: 1) Obtain the schedule's cron expression 2) Obtain the properties and arguments from the schedule. 3) Destroy/Create -or- Update the schedule (based on the platform's features).

To update a schedule SCDF would require new methods added to the Deployer SPI, CloudFoundry Deployer and, Kubernetes deployer to obtain the properties and command line arguments from the created schedules.

There is a planning meeting coming up and the team will discuss when this can be scheduled to be worked on. But contributions are welcome :-)

ntantri commented 3 years ago

Hi @cppwfs,

Appreciate the inputs on this.

Would be happy to contribute to this. Do let me know if I can propose a draft version of the changes :)

cppwfs commented 3 years ago

Absolutely ! and thank you