Netflix / conductor

Conductor is a microservices orchestration engine.
Apache License 2.0
12.81k stars 2.34k forks source link

Empty task list returned by poll APIs #3557

Open pallavisantosh19 opened 1 year ago

pallavisantosh19 commented 1 year ago

I'm using Conductor with Redis-Standalone persistence to test workflow executions. When I start a workflow, I see the workflow go into the RUNNING state and the next task being added to the task queue but I never see it become available to the worker.

The queue details API provides this - curl -X GET https://<host>/api/tasks/queue/all {"HTTP":0,"JOIN":0,"START_WORKFLOW":0,"SUB_WORKFLOW":0,"_deciderQueue":1,"simple-task-name":1}

however, when I attempt to poll tasks, I get an empty list with both the poll and batchPoll APIs

curl -X GET https://<host>/api/tasks/poll/simple-task-name < HTTP/2 204 < date: Wed, 29 Mar 2023 18:37:30 GMT

I do see Conductor logging the following - 2023-03-29 21:44:12,301 2517031 [http-nio-8080-exec-7] DEBUG [o:, u: t:] com.netflix.conductor.core.execution.WorkflowExecutor - Added task TaskModel{taskType='simple-task-name', status=SCHEDULED, inputData={data_by_section={BODY={channel=dpa-orchestrator, text=this is a test conductor task execution}}, referenceTaskName='simple-task-0', retryCount=0, seq=1, correlationId='null', pollCount=0, taskDefName='simple-task-name', scheduledTime=1680126252246, startTime=0, endTime=0, updateTime=0, startDelayInSeconds=0, retriedTaskId='null', retried=false, executed=false, callbackFromWorker=true, responseTimeoutSeconds=15, workflowInstanceId='dd001e4b-cd93-48b4-bc71-211941069546', workflowType='pallavi', taskId='6c3512f3-21fe-4fc9-ba93-97e6bc591c80', reasonForIncompletion='null', callbackAfterSeconds=0, workerId='null', outputData={}, workflowTask=simple-task-name/simple-task-0, domain='null', waitTimeout='0', inputMessage=null, outputMessage=null, rateLimitPerFrequency=0, rateLimitFrequencyInSeconds=1, externalInputPayloadStoragePath='null', externalOutputPayloadStoragePath='null', workflowPriority=0, executionNameSpace='null', isolationGroupId='null', iteration=0, subWorkflowId='null', subworkflowChanged=false} with priority 0 to queue simple-task-name with call back seconds 0

followed by this when i attempt to query the tasks 2023-03-29 21:51:16,783 3033041 [http-nio-8080-exec-1] DEBUG [o:, u: t:] com.netflix.conductor.service.TaskServiceImpl - The Tasks [] being returned for /tasks/poll/simple-task-name?null&null

Note: I was previously using Conductor with Postgres persistence and was able to successfully schedule and poll tasks.

v1r3n commented 1 year ago

Hi @pallavisantosh19 are you still facing the issue?

pallavisantosh19 commented 1 year ago

@v1r3n I temporarily resolved the issue by hard coding the redis rack value to "us-west-1a", although, I'm unaware if the implications of doing so. What is a rack and why does it need to be set to a specific AWS availability zone?