Open pawel-big-lebowski opened 2 years ago
@pawel-big-lebowski Love this idea! Added it to our roadmap. I'd be great to get a proposal written outlining how the web UI would display the notifications and any additional API changes needed to support aggregating the notification events in order to provide this level of transparency (similar to what you've outlined above). Would you be interested in writing the proposal?
Several things can happen within a dataset or a job:
As a data engineer I can think of plenty of such scenarios. We do have in Marqeuz all the information to detect it but it is not that easy to visualize them. As a a data engineer, with a limited web UI skills, it is hard for me to make use of this information although I know how valuable it is.
In each dataset/job view we could have something like
Notifications
where such information can be presented. This would allow other Marquez contributors to implement their own notifications or alerts.