Schedule Conductor workflow is a scheduler as a service that runs in the cloud with Netflix conductor embedded in it. It runs as an extension module of conductor.
PUT /scheduling/metadata/scheduleWf/{name} does not shutdown scheduler job on all servers.
Currently, this API marks scheduling metadata for a shutdown.
DefaultSchedulerManager reads to be shutdown metadata definitions.
cancel job applicable and clean up metadata.
In a cluster environment, metadata gets removed from database by the node which cancels the jobs first. The other node, then does not find metadata applicable for shutdown hence job keeps on running on other leftover nodes.
Proposed Fix:
Do not delete scheduled wf metadata definition after scheduled job cancel.
Add rescheduling capability in CronBasedWorkflowScheduler.
Meta definition will persist in DB even after cancellation of scheduled job (on each node of the conductor cluster).
SHUTDOWN and DELETE status of scheduled workflow metadata definition are complimentary to each other after this fix.
As of now, we do not need metadata definition clean-up. We will introduce DELETE API for metadata definition in the future major releases.
PUT /scheduling/metadata/scheduleWf/{name} does not shutdown scheduler job on all servers.
DefaultSchedulerManagerreads to be shutdown metadata definitions.In a cluster environment, metadata gets removed from database by the node which cancels the jobs first. The other node, then does not find metadata applicable for shutdown hence job keeps on running on other leftover nodes.
Proposed Fix:
CronBasedWorkflowScheduler.As of now, we do not need metadata definition clean-up. We will introduce DELETE API for metadata definition in the future major releases.