search

microsoft / durabletask-mssql

Microsoft SQL storage provider for Durable Functions and the Durable Task Framework
MIT License
87 stars 32 forks source link

Duplicate task execution #132

Open bhugot opened 1 year ago

bhugot commented 1 year ago

Hello, with a standard configuration of the provider, we juste found a use case where same activity was run twice without any trace in the history. Only trace we found was the double call in our appinsight. I will try to reproduce it. (the server was not at all under heavy stress)

cgillum commented 1 year ago

Were the duplicate calls on the same VM instance or different VM instances? Also, how much time was there between the two invocations. In most cases, this would indicate an automatic retry caused by a process failure.

bhugot commented 1 year ago

I think it's matching lock has the duration took about 2 min more than usual

bhugot commented 1 year ago

it's different VM instances

gsimardimt commented 7 months ago

Is this issue still active ? We also found a case of activities being run twice. Our function app is in consumption mode and the interval between the two invocations is about 5 minutes.

cgillum commented 7 months ago

Yes, still active. However, it should be noted that some duplicate task executions can be expected, particularly when VMs come in and out of rotation. This is especially true when running function apps in the consumption plan, where app instances can be created and removed within seconds. There is potentially some work that can be done to make the transitions more graceful and reduce the likelihood of duplicate executions. More investigation would be required to see what's possible.