Open dhiaayachi opened 1 month ago
This is a valid concern! Currently, Temporal workers only load the partition they are assigned to. This can lead to delays in dispatching tasks in low traffic situations, especially when combined with a long worker poll timeout.
Here are a couple of potential solutions:
Pre-loading all partitions: One approach could be to pre-load all partitions associated with the task queue when the worker starts. This would eliminate the need for additional network calls to load specific partitions, improving dispatch latency in low traffic scenarios.
Dynamically fetching relevant partitions: Alternatively, Temporal could dynamically fetch additional partitions based on load or activity. When a worker is idle for a certain duration, it could proactively fetch partitions that are experiencing high load, ensuring the most efficient distribution of tasks.
Implementing either of these solutions would offer several benefits:
Before implementing such a feature, it's important to consider the following:
Loading all task queue partitions or dynamically fetching relevant partitions could significantly improve task dispatch efficiency, especially in low traffic scenarios. Further research and testing are required to evaluate the potential impact and select the most suitable approach.
Related Documentation:
Thank you for the feature request! It's a great idea to have all task queue partitions loaded to prevent delays in task dispatch, especially in low traffic situations.
Currently, Temporal doesn't have this feature built-in. However, you can use the temporal task-queue list-partition
command to get the list of partitions for a specific task queue and then manually configure your workers to connect to all of them.
For more information on how to manage task queues, you can refer to Temporal CLI task-queue command reference.
Thank you for the feature request!
Currently, Temporal doesn't have a feature to load all task queue partitions when a root partition is loaded. This is a great idea, as it could reduce delays in task dispatching in low traffic situations, especially with long worker poll timeouts.
You can work around this limitation by increasing the number of worker processes for your task queue. This way, you ensure that there are always enough workers available to process tasks, even in situations where there is low traffic and the worker poll timeout is long.
We appreciate your feedback and will consider adding this feature to Temporal in the future!
Thank you for the feature request! This would help prevent potential delays in task dispatch.
Currently, Temporal doesn't offer this feature. However, to work around this, you could increase your worker poll timeout value which will lessen the chances of any delays in task dispatch. You can also consider using the temporal task-queue list-partition
command to view the partitions of your task queue.
Thank you for the feature request! This is a great suggestion.
Currently, Temporal does not automatically load all partitions of a task queue when the root partition is loaded. This is because loading all partitions can be computationally expensive and can lead to performance degradation, especially in high-traffic scenarios.
As a workaround, you can manually load all partitions of a task queue using the Temporal CLI command temporal task-queue list-partition
. This command will list all partitions of a task queue, and you can then use the temporal task-queue describe
command to load the partitions individually.
This approach will allow you to achieve the desired functionality without impacting overall system performance.
Thank you for the feature request! Currently, there's no way to load all task queue partitions at once. You can use the temporal task-queue list-partition
command to list all partitions and then use the temporal worker
command with the --task-queue
flag to start workers for each of the partitions.
Feature request to load all task queue partitions when root one is loaded. This helps prevent possible delays in task dispatch in low traffic situations and with the long worker poll timeout.