Netflix / conductor

Conductor is a microservices orchestration engine.
Apache License 2.0
12.81k stars 2.34k forks source link

Task does not time out after retrying once #3864

Open zhouwanbin opened 1 year ago

zhouwanbin commented 1 year ago

Describe the bug Task does not time out after retrying once

Details Conductor version: v3.13.3 Persistence implementation: MySQL Queue implementation: MySQL Lock: Redis Workflow definition: { "createTime": 1701072417382, "accessPolicy": {}, "name": "InputOutputDemo017", "description": "weather warning description", "version": 4, "tasks": [ { "name": "dt_taskTA_006", "taskReferenceName": "dt_taskTA_006_reference011", "inputParameters": { "city": "${workflow.input.city}" }, "type": "SIMPLE", "startDelay": 0, "optional": false, "asyncComplete": false, "reuse": false }, { "name": "dt_taskB_006", "taskReferenceName": "dt_taskB_006_reference011", "inputParameters": { "receiver": "${dt_taskTA_006_reference011.output.outputKey}", "content": "${dt_taskTA_006_reference011.output.oddEven}" }, "type": "SIMPLE", "startDelay": 0, "optional": false, "asyncComplete": false, "reuse": false } ], "inputParameters": [], "outputParameters": {}, "schemaVersion": 2, "restartable": true, "workflowStatusListenerEnabled": false, "ownerEmail": "zwb@test.com", "timeoutPolicy": "TIME_OUT_WF", "timeoutSeconds": 86400, "variables": {}, "inputTemplate": {}, "business": "develop", "affiliated": false }

Task definition:

{ "createTime": 1701067907104, "createdBy": "", "accessPolicy": {}, "name": "dt_taskTA_006", "retryCount": 1, "timeoutSeconds": 90, "inputKeys": [ "city" ], "outputKeys": [ "outputKey", "oddEven", "mod" ], "timeoutPolicy": "RETRY", "retryLogic": "FIXED", "retryDelaySeconds": 10, "responseTimeoutSeconds": 60, "inputTemplate": {}, "rateLimitPerFrequency": 0, "rateLimitFrequencyInSeconds": 1, "ownerEmail": "zwb@test.com", "backoffScaleFactor": 1, "dailyQuota": 0, "business": "develop" }

{ "createTime": 1701067907402, "createdBy": "", "accessPolicy": {}, "name": "dt_taskB_006", "retryCount": 1, "timeoutSeconds": 90, "inputKeys": [ "receiver", "content" ], "outputKeys": [ "outputKey", "oddEven", "mod" ], "timeoutPolicy": "RETRY", "retryLogic": "FIXED", "retryDelaySeconds": 10, "responseTimeoutSeconds": 60, "inputTemplate": {}, "rateLimitPerFrequency": 0, "rateLimitFrequencyInSeconds": 1, "ownerEmail": "zwb@test.com", "backoffScaleFactor": 1, "dailyQuota": 0, "business": "develop" }

To Reproduce Steps to reproduce the behavior:

  1. The implementation of dt_taskTA_006 and dt_taskB_006 is sleep 300 second.

Expected behavior The task dt_taskTA_006 runs twice and times out each time. Final workflow times out

Screenshots The actual results are as follows image

Additional context