Azure / azure-functions-durable-extension

Durable Task Framework extension for Azure Functions
MIT License
713 stars 270 forks source link

Enchance out-of-proc schema to simplify future out-of-proc SDKs #1662

Open davidmrdavid opened 3 years ago

davidmrdavid commented 3 years ago

Is your feature request related to a problem? Please describe. Our current out-of-proc schema does not contain enough information to confidently guarantee the correctness, and determinism, of our out-of-proc SDKs. Currently, the JS and Python implementations have a rather defensive implementation that attempts to minimize bugs, but some definitely remain.

In the current state of affairs, the upcoming Durable SDKs are likely to suffer from the same limitations. The earlier we improve our schema, the earlier those SDKs can harvest a more robust implementation.

Describe the solution you'd like I'm unsure of the specifics, as this will require some whiteboarding. Regardless, any appropiate solution should improve the performance of our event-lookup algorithms while also providing sufficient event metadata to deterministically look-up events when actions are processed.

Also, an appropiate solution would ideally also allow us to gradually migrate to this new schema, as to avoid prompting an urgent, error-prone, re-implementation of existing SDKS. In other words, ideally this new schema will be backwards compatible.

Describe alternatives you've considered N/A

Additional context More Durable SDKs are in the works, so this task is of relative urgency.

davidmrdavid commented 3 years ago

FYI @ConnorMcMahon: let's prioritize this for our next extension release