[Improvement] Gravitino UI takes 4-10 seconds to display list of catalogs

datastrato / gravitino

World's most powerful data catalog service with providing a high-performance, geo-distributed and federated metadata lake.

https://datastrato.ai/docs/

Apache License 2.0

347 stars 150 forks source link

[Improvement] Gravitino UI takes 4-10 seconds to display list of catalogs #3244

Open justinmclean opened 2 weeks ago

justinmclean commented 2 weeks ago

What would you like to be improved?

When using the playground it can take longer than expected time to display the list catalogs, this didn't use to be the case. This occurs every time and not just the first time.

How should we improve?

Improve the time and/or cache the list of catalogs.

shaofengshi commented 1 week ago

+1, the loading time is longer than expected.

jerryshao commented 5 days ago

I guess the problem is that UI now changes to list all the catalog info, rather than name when listing, this takes longer time than before. @mchades can yu please help to check this, to see how we can improve the performance here?

jerryshao commented 5 days ago

The problem I guess is due to catalog load, because we have to filter out hidden value, which needs to create all the classloader each time, this is very time consuming. @mchades can you please investigate on this to figure out a better solution.