PrefectHQ / graphs

Large scale graphs designed for Prefect
Apache License 2.0
9 stars 1 forks source link

Render events with their respective tasks #467

Open brandonreid opened 5 months ago

brandonreid commented 5 months ago

When adding the Events layer, a couple different approaches were taken and accommodations for scale both on the front and backend API level lead us to determine that a separate events request was ideal. This lead to simply showing all events at the flow level.

For events that are associated with tasks, the architecture of the graphs repo requires the events data to be accessible from the node. This is important for a number of reasons, one being that it prevents every node from needing to look up whether events are applied to them or not on every render, instead they can simply call a render function for any events they may have in accordance with their normal cache check. So somewhere in the process we would ideally layer the task related event data onto the node. That's the current thinking anyway, maybe there is a way to create an efficient lookup for node related events.

For layering the events onto the node level data, the ideal place to do this, in order to avoid looping through the data as much as possible, would be during the mapping step in prefect-ui-library. When requesting data, we'd request the graph data and the events at the same time, then map the graph data, applying related task events at that time. This requires indexing events in a way that allows for fast lookups, possibly a new end point for this even. Each event has a number of related resources, but we should be able to reduce them to just task resources for the sake of lookups.