There is a need to improve the speed of updating the Kedro-Viz UI when using the autoreload flag to below 10 seconds. A rendering time of 35 seconds is a pain point, and can create a performance perception.
Context
This was highlighted in #1750, and also by a slack user -
"In your experience, do you think that these warnings can slow down the rendering of Kedro visualization? It takes about 35 seconds to render, and my project is not that big yet. Is 35 seconds a typical time to render?"
"Yes I'm aware of autoreload! Thanks. But the reload takes a long time too! I have PickleDatasets, CSVDataset, ParquetDataset and a custom dataset that loads and saves SentenceTransformer models."
Description
There is a need to improve the speed of updating the Kedro-Viz UI when using the autoreload flag to below 10 seconds. A rendering time of 35 seconds is a pain point, and can create a performance perception.
Context
This was highlighted in #1750, and also by a slack user -
"In your experience, do you think that these warnings can slow down the rendering of Kedro visualization? It takes about 35 seconds to render, and my project is not that big yet. Is 35 seconds a typical time to render?"
"Yes I'm aware of autoreload! Thanks. But the reload takes a long time too! I have PickleDatasets, CSVDataset, ParquetDataset and a custom dataset that loads and saves SentenceTransformer models."