Optimize large payload fan-out to activities and child workflows

[mfateev]

Is your feature request related to a problem? Please describe. Some use cases need to pass the same payload to a large number of activities or child worklfows. In the current model, the payload will be repeated in the history as many times as the number of invocations. Recording the same payload many times is clearly nonoptimal.

Describe the solution you'd like I don't have a specific API/feature proposal to resolve this issue. But I believe some solution that saves such a payload only once should be possible.

One simplistic strawman is to have a special command to save a payload and then allow referencing it in other places of a workflow.

Another idea is to perform payload deduplication by storing large payload hashes in the mutable state.

Additional context A forum post that describes a customer having issues with a large payload passed to many child workflows.

[mycroftcanner]

I wanted to add that the idea of "payload deduplication by storing large payload hashes in the mutable state" mentioned in the GitHub issue reminds me of Plan 9's Venti. This approach could also potentially enforce a write-once policy, preventing accidental or malicious destruction of data.