search

NCEAS / metacatui

MetacatUI: A client-side web interface for DataONE data repositories
https://nceas.github.io/metacatui
Apache License 2.0
42 stars 27 forks source link

Use Solr 7 features to minimize queries #1356

Open laurenwalker opened 4 years ago

laurenwalker commented 4 years ago

Look into Solr features released after version 3 (metacat 2.12.0 and earlier) to minimize query HTTP calls sent by MetacatUI views.

Some examples and the Solr join query parser and subqueries.

Since this requires Metacat 2.13.0, we should be careful about when to release this in MetacatUI since users who haven't upgraded won't be able to move forward that UI release.

Places for improvement:

DataCatalogView

  1. Query to get the 25 search results
  2. Using the results of that query, the view sends a list of all resourceMap pids to get any package members that have provenance info indexed, for the prov icon in the search result rows.
  3. Query to get earliest beginDate (for year filter slider)
  4. Query to get latest endDate (for year filter slider)

Request 1 and 2 could be combined. Request 3 and 4 could possibly be removed all together or only sent when the Year filter is expanded. I don't think we need to update the year slider with every single search since that filter is used infrequently.

MetatdataView

  1. Solr query sent to get the science metadata doc, particularly to find the resourcemap pid
  2. Using the resource map pid, get the data files in the package to build the package contents table. Request 2 above has to wait for Request 1 to finish. These should be combined into a single query so the package contents table loads faster. (Needs performance testing, but I'm guessing it's faster to do one GET for both)

StatsView (used by portals) This is partially getting addressed in #1346. There are 9 total queries sent to Solr to render the various graphs, which I won't go into right now, but they should be looked at.

laurenwalker commented 4 years ago

In #1346, the Stats model was refactored, so no work is needed for the StatsView:

StatsView (used by portals) This is partially getting addressed in #1346. There are 9 total queries sent to Solr to render the various graphs, which I won't go into right now, but they should be looked at.