PrefectHQ / prefect

Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
https://prefect.io
Apache License 2.0
15.95k stars 1.57k forks source link

Automation: Ability for result of a completed Flow Run trigger to be passed to a Deployment Run action's parameters #11718

Open discdiver opened 8 months ago

discdiver commented 8 months ago

First check

Prefect Version

2.x

Describe the current behavior

You must use run_deployment, a webhook, or a emit_event to populate the parameters of a Run Deployment automation action.

Describe the proposed behavior

It would be convenient if you could pass the results of a completed flow run into the Run Deployment's parameters via an automation.

Example Use

The situation where you want to match on one of several flows is a use case where this pattern might be particularly useful.

Additional context

This would require some thinking around result persistence and would only apply to flow run triggers with a completed state.

@WillRaphaelson had some thoughts around how implementation might work.

WillRaphaelson commented 8 months ago

This is a super interesting test case for some of the product approach here - I wonder if we should do this, should it be a special case for completed states, or should we store or link to serialized results in all state change events. Zooming out, what does a user get from doing this serverside via automations rather than chaining flows client side? cc @chrisguidry just an interesting use case.