Solr Performance: Lazy-load some facets

[cdrini]

Facets are expensive and have performance implications. Make some of the facets collapsed by default, and only load in when the user expands them.

Requires adding a way to fetch facets from a search endpoint.

Initial Solution

For now, the path forward is (perhaps led by @jimchamp):

• [ ] Make /search page facets an async partial called facets.html
  • Clean & re-organize the facet code + move to its own template
  • /search loads facets.html w/ headers & no data (showing loading bars)
  • After page load, /search makes 1 call to the /partials server to re-fetch facet.html w/ data
  • If facets fail, facet.html partial should not show loading bars

Phases

1. Simply move the facet sidebar into its own template with loading and defaults (even if we’re not using them in step 1.
2. Refactor the /plugins/worksearch/code.py so it’s possible to make 2 separate queries: facet counts (no results) and results (no facet counts) while preserving ability to support facet counts + results
3. Create partial which returns facets template w/ no results, just facet counts… Update the main template to use results, no facet counts

Polished Solution

This should probably be led by @cdrini

• [ ] Extend the search.json API with a facets option to return counts
  • this will enable /search to make a 2nd async request for facet counts after page load that we can use to populate results into each facet section of the sidebar
• [ ] Update js + templates to power /search with async /search.json?facets

[mekarpeles]

In our https://github.com/internetarchive/openlibrary/blob/master/openlibrary/plugins/worksearch/code.py#L495-L502 that powers the /search endpoint, we ultimately do a do_search which uses run_solr_query to generate search results and facets for the work_search.html template (otherwise if there are no params, we fallback to an empty SearchResponse object).

We presumably want to control do_search from the search GET so that facets are False.

This will allow us to populate the search page without the facets initially.

cc: @cdrini we may need your help figuring out how to create or update the search API / endpoints to only return facet counts (no search results)

[cdrini]

^ Set the rows=0 ; on the do_search; that's the solr recommended way to only get facet data :+1: