search

uber / cadence-java-client

Java framework for Cadence Workflow Service
https://cadenceworkflow.io
Other
143 stars 106 forks source link

Infinite retry in workflow/activity worker on non-retryable error #249

Closed meiliang86 closed 5 years ago

meiliang86 commented 5 years ago

Currently WorkflowWorker and ActivityWorker retries on all error types, causing it to go into infinite retry loop if certain non-retryable error is encountered. For example, decision timeout.

2019-01-16 12:11:53,762 WARN [null build-tasks : 4] c.u.c.i.c.Retryer [Retryer.java:115] Retrying after failure
com.uber.cadence.BadRequestError: Expired decision: expectedHistorySize=32, actualHistorySize=36
    at com.uber.cadence.internal.testservice.TestWorkflowMutableStateImpl.lambda$completeDecisionTask$2(TestWorkflowMutableStateImpl.java:287)
    at com.uber.cadence.internal.testservice.TestWorkflowMutableStateImpl.update(TestWorkflowMutableStateImpl.java:201)
    at com.uber.cadence.internal.testservice.TestWorkflowMutableStateImpl.completeDecisionUpdate(TestWorkflowMutableStateImpl.java:185)
    at com.uber.cadence.internal.testservice.TestWorkflowMutableStateImpl.completeDecisionTask(TestWorkflowMutableStateImpl.java:279)
    at com.uber.cadence.internal.testservice.TestWorkflowService.RespondDecisionTaskCompleted(TestWorkflowService.java:335)
    at com.uber.cadence.internal.sync.TestWorkflowEnvironmentInternal$WorkflowServiceWrapper.RespondDecisionTaskCompleted(TestWorkflowEnvironmentInternal.java:653)
    at com.uber.cadence.internal.worker.WorkflowWorker$TaskHandlerImpl.lambda$sendReply$0(WorkflowWorker.java:273)
    at com.uber.cadence.internal.common.Retryer.lambda$retry$0(Retryer.java:75)
    at com.uber.cadence.internal.common.Retryer.retryWithResult(Retryer.java:93)
    at com.uber.cadence.internal.common.Retryer.retry(Retryer.java:72)
    at com.uber.cadence.internal.worker.WorkflowWorker$TaskHandlerImpl.sendReply(WorkflowWorker.java:273)
    at com.uber.cadence.internal.worker.WorkflowWorker$TaskHandlerImpl.handle(WorkflowWorker.java:242)
    at com.uber.cadence.internal.worker.WorkflowWorker$TaskHandlerImpl.handle(WorkflowWorker.java:221)
    at com.uber.cadence.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:71)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
meiliang86 commented 5 years ago

Fixed.