search

Azure / durabletask

Durable Task Framework allows users to write long running persistent workflows in C# using the async/await capabilities.
Apache License 2.0
1.53k stars 296 forks source link

Orchestration Tag support for SQL Provider #663

Open jundayin opened 2 years ago

jundayin commented 2 years ago

Brought up here for more visibility. https://github.com/microsoft/durabletask-mssql/issues/61

Currently SQL Provider seems does not support storing orchestration tag data. Right now we want to have some data to be shared among the orchestration to activity but we can only make it passed through input and output arguments one by one which is not error prone.

Would like to know is this feature on road map?

cgillum commented 2 years ago

Would like to know is this feature on road map?

Which feature are you referring to, specifically? The SQL provider issue will be worked on, but I don't think that will help you share data between orchestrations and activities beyond what is already supported today (inputs & outputs).

jundayin commented 2 years ago

@cgillum In DurtableTask.Core there is a public API Task<OrchestrationInstance> CreateOrchestrationInstanceAsync(string name, string version, string instanceId, object input, IDictionary<string, string> tags) which allows dev create orchestration event with tags data.

In SQL Provider tag is always stored null. (Code). It may not be directly support sharing data between orchestrations and activities but if the tag is available we can do something upon it. AFAIK some projects already utilize the tag as well as the ExtensionData field in OrchestrationInstance to convey the data across orchestrations and activities. https://github.com/jviau/durabletask-hosting/blob/master/samples/DurableTask.Samples/SessionData/OrchestrationInstanceEx.cs . But this one is yet achievable by SQL Provider

cgillum commented 2 years ago

Thanks for the pointer - I haven't seen this pattern of sharing arbitrary metadata before. Unfortunately, it's a bit of a hack and this approach of extending OrchestrationInstance through ExtensionData won't work for the SQL provider because the SQL provider doesn't use serialization to construct or persist these objects. It will probably work fine for the Azure Storage and Service Bus providers, though. I think the best way to make this work consistently for all providers is to come up with a new feature/protocol for sharing ambient data.

jundayin commented 2 years ago

I see. So it needs more discussion and it may not be on the table in short term, right? Just to confirm so that I can close the option inside my team and we will go for input/output approach

cgillum commented 2 years ago

Correct - more discussion will be needed, and there aren't any short-term options available, unfortunately.