iWF is a WorkflowAsCode microservice orchestration platform offering an orchestration coding framework and service for building resilient, fault-tolerant, scalable long-running processes
MIT License
543
stars
55
forks
source link
[bug]ConditionalComplete improvement: also check newly returned channel messages from state APIs #289
In the first iteration of conditionalComplete in https://github.com/indeedeng/iwf/issues/282, it may lose the messages published to the InternalChannel from State APIs. This could happen when the internalChannelPublishing is from StateApi response that are returned within the same Cadence/Temporal Workflow[Decision]TaskCompleted batch.
As workaround: user need to make sure that they don't have workflow state publishing the internal messages when using this ConditionalComplete feature,
As workflow 2: user uses persistence locking, to let the state API lock on the same field so that they won't execute at the same time(hence won't have the internalChannelPublishing in the same Cadence/Temporal Workflow[Decision]TaskCompleted batch.
There are a couple ways to "fix" or improve this behavior. For example, let the thread of having ConditionalComplete to "temporarily yield" so that other threads can execute, hence other threads can process the internalChannelPublishing from StateApi response.
To implement the temporarily yield, the current thread can open a new thread, which is to set a flag to true. And the current thread uses await to wait for the flag to be true. This will enforce the current thread to yield and then come back again when all other thread got a chance to run.
longquanzheng
commented
1 week ago
In the first iteration of conditionalComplete in https://github.com/indeedeng/iwf/issues/282, it may lose the messages published to the InternalChannel from State APIs. This could happen when the internalChannelPublishing is from StateApi response that are returned within the same Cadence/Temporal Workflow[Decision]TaskCompleted batch.
As workaround: user need to make sure that they don't have workflow state publishing the internal messages when using this ConditionalComplete feature, As workflow 2: user uses persistence locking, to let the state API lock on the same field so that they won't execute at the same time(hence won't have the internalChannelPublishing in the same Cadence/Temporal Workflow[Decision]TaskCompleted batch.
There are a couple ways to "fix" or improve this behavior. For example, let the thread of having ConditionalComplete to "temporarily yield" so that other threads can execute, hence other threads can process the internalChannelPublishing from StateApi response.
To implement the temporarily yield, the current thread can open a new thread, which is to set a flag to true. And the current thread uses
awaitto wait for the flag to be true. This will enforce the current thread to yield and then come back again when all other thread got a chance to run.