Add a rate limit option for batch operations

[dhiaayachi]

Is your feature request related to a problem? Please describe.

Batch operations are run server side and may effect millions of executions, this in turn may overload workers and disrupt normal operations.

Describe the solution you'd like

Allow rate limiting a batch operation, this option should be allowed for all types of batch operations.

Additional context

The previous batch operation implementation was client side in tctl and it had an --rps (and `--concurrency) option, we took that ability away from users when we ported the implementation to the server.

[dhiaayachi]

Thank you for the feature request. We appreciate you bringing this to our attention.

Currently, there's no way to directly limit the rate of batch operations in Temporal. However, you can achieve a similar effect by:

• Breaking down large batch operations: You can divide a large batch into smaller ones, running them sequentially with a delay between each. This gives your workers time to catch up.
• Increasing worker capacity: If the batch operations are frequently overloading your workers, consider increasing the number of workers in your deployment to handle the load more effectively.

We will consider adding rate limiting to batch operations in future releases. Feel free to follow the discussion on our community forum or GitHub to stay updated on the progress: https://community.temporal.io/

[dhiaayachi]

Thank you for your feature request. We understand that the lack of rate limiting for batch operations can be problematic. You can currently use the --rps flag in the tctl batch start command to work around this.

[dhiaayachi]

Thank you for your feature request!

We understand the need for rate limiting batch operations to prevent overloading workers. Currently, Temporal CLI doesn't provide a specific rate limiting option for batch operations. As a workaround, you could try to achieve similar functionality by:

• Adjusting the --rps flag when starting the batch job: This flag controls the requests per second for processing the batch job. By lowering the value, you can effectively limit the rate of batch operations.
• Implementing a custom client-side solution: You could write a custom client-side script that sends batch requests with a specific delay or uses a token bucket algorithm to control the rate.

We appreciate your feedback and will consider adding rate limiting as a feature in future releases.

[dhiaayachi]

Thank you for your feature request. We understand the need for rate limiting batch operations to prevent worker overload.

While rate limiting for batch operations is not currently available, you can consider these workarounds:

• Use the --rps flag: The tctl CLI used to have an --rps option for rate limiting batch operations on the client-side. While it is no longer available in the server-side implementation, you can still use it to control the rate at which tctl performs the batch operation. This approach gives you some level of control but might not be as fine-grained as server-side rate limiting.
• Use a more specific query: If possible, narrow down your query to target fewer workflow executions. This will reduce the overall impact of the batch operation on your workers.
• Split your batch operations: Consider breaking your batch operations into smaller chunks that target fewer workflow executions at a time. This will allow you to more evenly distribute the load across your workers.

We appreciate your feedback and will consider adding this feature in the future.