PrefectHQ / prefect

Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
https://prefect.io
Apache License 2.0
15.31k stars 1.5k forks source link

Add short doc on transactions #14258

Closed cicdw closed 4 days ago

cicdw commented 5 days ago

This doc is imperfect, but hopefully establishes a good first step in documenting transactions. I'll continue to refine in follow up PRs if accepted.

billpalombi commented 5 days ago

I see that this deletes the page on retries. Is that intentional?

cicdw commented 5 days ago

@billpalombi yea that was intentional, I didn't see why retries needed their own page and the title felt overblown relative to the content.

billpalombi commented 4 days ago

yea that was intentional, I didn't see why retries needed their own page and the title felt overblown relative to the content.

Agreed on the title. I do think the Custom retry behavior, jitter, and global configuration content should live somewhere though.

I see at least two options:

cicdw commented 4 days ago

Yea agree those should be documented, there's just a small surface area - the task and flow pages make the most sense to me (and there are non-trivial distinctions in those docs - flows need to discuss infra retries, how task reruns are handled, etc). I can work on that doc next after I do another pass through of some caching functionality.

billpalombi commented 4 days ago

@cicdw are you already working on adding the retry content to the flow and task pages? If not, I will do that now. I don't want to forget that.

cicdw commented 4 days ago

@billpalombi nope haven't picked that up yet, take it away