Add new Workflow Id Reuse Policy: Allow Duplicate with Queueing

[dhiaayachi]

Is your feature request related to a problem? Please describe. N/A

Describe the solution you'd like Currently, when a duplicate Workflow ID is submitted and the "Reject Duplicate" policy is used (default), the SDK raises an exception, which is the desired behavior. It would be useful to add a new reuse policy like "Queue Duplicate" where the workflow would be queued (with a new state like "Queued").

Describe alternatives you've considered The alternative is to use the "master-workflow + signal + child-workflow" recipe, which reduces the observability (i.e. the UI can't be used to see what workflows are queued) and increases the design complexity for simple use-cases.

Additional context N/A

[dhiaayachi]

Thank you for the feature request! This is a great idea for simplifying the process of handling duplicate workflow IDs.

Currently, Temporal doesn't have a "Queue Duplicate" policy built in. However, you can achieve a similar result by using a combination of a dedicated task queue and a Signal workflow pattern.

1. Dedicated Task Queue: Create a separate task queue specifically for queued workflows.
2. Signal Workflow: Create a workflow that is responsible for handling the queued workflows. This workflow can be signaled when a duplicate workflow ID is submitted.
3. Workflow Queueing: When a duplicate workflow ID is submitted, instead of raising an exception, you can signal the dedicated workflow queue. The signal can carry the workflow input, any other necessary context, and the original workflow ID.
4. Processing Queued Workflows: The signal workflow will process the queued workflows one by one. Once a workflow is dequeued, it can be started with the original workflow ID or a new one if you want to prevent further queuing.

This approach will allow you to effectively queue workflows that would otherwise be rejected due to duplicate IDs, while maintaining a degree of observability through the signal workflow's status.

Let me know if you have any other questions!

[dhiaayachi]

Thank you for the feature request!

Currently, Temporal doesn't have a "Queue Duplicate" policy built-in. You can achieve a similar behavior using a combination of the "Reject Duplicate" policy with a custom retry policy for your Workflow.

Here's a potential approach:

• Use the "Reject Duplicate" policy to prevent duplicate Workflow executions.
• Implement a retry policy with a backoff strategy that will retry the Workflow execution after a certain delay.

This way, if a duplicate Workflow ID is submitted, Temporal will reject the request, and your Workflow will retry after a designated delay. This can be useful to ensure that all tasks are eventually processed.

Please let me know if you have any other questions.

[dhiaayachi]

Thank you for the feature request! This is a great suggestion for making it easier to manage duplicate workflows.

While we don't have this functionality built-in yet, you can achieve a similar effect using the "master-workflow + signal + child-workflow" pattern you mentioned. While this adds some complexity, it allows you to maintain control over the queued workflows:

1. Master Workflow: Create a master workflow that receives the workflow requests and checks for duplicates.
2. Queueing: If a duplicate is detected, the master workflow can add the request to a queue (e.g., a Redis list) and signal a child workflow.
3. Child Workflow: The child workflow listens for signals from the master and processes requests from the queue.

This approach provides a way to handle duplicate workflows while maintaining observability and flexibility. Let us know if you have any other questions.