[Core feature] Add async API for getting entire workflow state including node execution

[pmahindrakar-oss]

Motivation: Why do you think this is important?

As a flyte API user for getting execution details, they have to be aware about following additional API's

• ListNodeExecutions
• ListTaskExecutions
• GetNodeExecutionData

Along with the regular GetExecution API Also there is structure to how these API calls are to be made based on the parent child relationships between the node executions and also if a node is dynamic or subworflow node.

This complexity is implemented in all the clients right now including UI and the CLI.

Would be great to support one async API to gather all this info on the Admin side.

Create a async process in admin, that consolidates the entire state into (a set of ) protobuf messages (that can be chained) on completion of an execution. The API, then should return this object. For completed workflows the return can be really fast. For running workflows, we can actually perform the queries. Another option could be to lazily cache as workflows are invoked

Goal: What should the final outcome look like, ideally?

Async Admin API to trigger gather of overall state of the execution and using the same or different API to get the state details

Describe alternatives you've considered

Current way exists to gather this info in UI and CLI which is complex

Propose: Link/Inline OR Additional context

No response

Are you sure this issue hasn't been raised already?

• [X] Yes

Have you read the Code of Conduct?

• [X] Yes

[pmahindrakar-oss]

cc : @kumare3

[EngHabu]

Let's plan to have a design doc reviewed by UI, and flytekit and flytectl owners by end of March. @pmahindrakar-oss

[github-actions[bot]]

Hello 👋, This issue has been inactive for over 9 months. To help maintain a clean and focused backlog, we'll be marking this issue as stale and will close the issue if we detect no activity in the next 7 days. Thank you for your contribution and understanding! 🙏

[github-actions[bot]]

Hello 👋, This issue has been inactive for over 9 months and hasn't received any updates since it was marked as stale. We'll be closing this issue for now, but if you believe this issue is still relevant, please feel free to reopen it. Thank you for your contribution and understanding! 🙏

[github-actions[bot]]

Hello 👋, this issue has been inactive for over 9 months. To help maintain a clean and focused backlog, we'll be marking this issue as stale and will engage on it to decide if it is still applicable. Thank you for your contribution and understanding! 🙏