The marquez-web application currently fetches data from the API only at initialization time instead of just-in-time. The application currently queries 700 jobs and datasets. This limits our ability to make larger amounts of data captured by the Marquez API available for visualization in the web application.
Our filtering operations also only modify the initial set of data bought down from the server. Filtering operations should be making asynchronous calls to the API to expose all of the Marquez data and allow the application to scale to larger sets of data.
The
marquez-webapplication currently fetches data from the API only at initialization time instead of just-in-time. The application currently queries 700 jobs and datasets. This limits our ability to make larger amounts of data captured by the Marquez API available for visualization in the web application.Our filtering operations also only modify the initial set of data bought down from the server. Filtering operations should be making asynchronous calls to the API to expose all of the Marquez data and allow the application to scale to larger sets of data.