flyteorg / flyte

Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks.
https://flyte.org
Apache License 2.0
5.45k stars 582 forks source link

[Core Feature] Enriched egress events from flyteadmin #1019

Open tnsetting opened 3 years ago

tnsetting commented 3 years ago

Motivation: Why do you think this is important? Currently raw flyte execution events can be configured to publish to pub/sub. In order to make these event more useful to downstream consumer, we will perform enrichment to these events. There are a few use case for these event such as data lineage, event based triggering.

Goal: What should the final outcome look like, ideally? The enriched events are published to pub/sub so downstream users can consume them.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

[Optional] Propose: Link/Inline OR Additional context If you have ideas about the implementation please propose the change. If inline keep it short, if larger then you link to an external document.

Linking this RFC here too: https://docs.google.com/document/d/1YMZkJr77Kg0IIJssFxs-QFMf2mpSEHIIP0hTVokqywY/edit Lack of progress on this ticket was driven partly by a lack of demand. Proper DataHub integration should be a good guiding/motivating example.

github-actions[bot] commented 11 months ago

Hello πŸ‘‹, This issue has been inactive for over 9 months. To help maintain a clean and focused backlog, we'll be marking this issue as stale and will close the issue if we detect no activity in the next 7 days. Thank you for your contribution and understanding! πŸ™

github-actions[bot] commented 11 months ago

Hello πŸ‘‹, This issue has been inactive for over 9 months and hasn't received any updates since it was marked as stale. We'll be closing this issue for now, but if you believe this issue is still relevant, please feel free to reopen it. Thank you for your contribution and understanding! πŸ™

github-actions[bot] commented 1 month ago

Hello πŸ‘‹, this issue has been inactive for over 9 months. To help maintain a clean and focused backlog, we'll be marking this issue as stale and will engage on it to decide if it is still applicable. Thank you for your contribution and understanding! πŸ™