camunda / zeebe

Distributed Workflow Engine for Microservices Orchestration
https://zeebe.io
3.05k stars 549 forks source link

fix: retry SetVariable on for UNAVAILABLE and RESOURCE_EXHAUSTED #18180

Closed berkaycanbc closed 1 week ago

berkaycanbc commented 2 weeks ago

Description

SetVariable endpoint is making changes on the state of the process instance. Therefore, it is not idempotent. If we retry setting variables in different stages of process instance, it might provide different behaviour. Because of that we shouldn't retry SetVariables on timeouts (DEADLINE_EXCEEDED) where the processing of the engine can continue, so the process instance state changes. For UNAVAILABLE and RESOURCE_EXHAUSTED, the broker will not accept further commands, so that when we retry the SetVariables command, the process instance state will stay the same.

Related issues

closes #15549

berkaycanbc commented 2 weeks ago

@oleschoenburg @korthout We worked on this issue during our mob-programming session. I added our ideas in the commit message (same as in the PR description). We added you as reviewers since Nico reviewed the original PR https://github.com/camunda/zeebe/pull/15230 and Ole created the issue for this PR. Please take a look. 👀

korthout commented 1 week ago

@berkaycanbc we should consider whether we want to backport this

berkaycanbc commented 1 week ago

@berkaycanbc we should consider whether we want to backport this

Thanks @korthout for reminding, I will apply backports for the two latest releases. 👍

berkaycanbc commented 1 week ago

/backport

backport-action commented 1 week ago

Successfully created backport PR for stable/8.4:

backport-action commented 1 week ago

Successfully created backport PR for stable/8.5: