Improve TaskHandler polling strategy

conductor-sdk / conductor-python

Conductor OSS SDK for Python programming language

Apache License 2.0

62 stars 29 forks source link

Open gardusig opened 2 years ago

gardusig commented 2 years ago

Add new parameter for batchSize
Each worker will have a standalone subprocess to poll indefinitely
- For each polled task, start a new subprocess to execute and update it

hebrd commented 2 years ago

It would be great if subprocess count for each task could be configured independently, like TaskThreadCount in JAVA client