internetarchive / openlibrary

One webpage for every book ever published!
https://openlibrary.org
GNU Affero General Public License v3.0
5.25k stars 1.39k forks source link

Lazily make carousel queries when a patron encounters a carousel #5728

Open ghost opened 3 years ago

ghost commented 3 years ago

This issue is not about lazy loading the items inside each carousel. We already do that, as each carousel limits the number of results from its search query to a default of 20. Patrons can click an arrow to "load more," which works great.

However, the collections pages still load extremely slowly. For example, the LGBTQ collection contains 34 carousels and more than a thousand books, and the page as a whole takes 10.24 seconds to load. I investigated in the Network tab and saw that every carousel on a page makes its search query immediately upon the initial page load.

This is a screenshot of my network tab. I'm on Linux Mint 20, Firefox for Linux Mint 91.0.2. ![current-network-tab](https://user-images.githubusercontent.com/85025703/136041266-e046e264-e9ee-43dd-b156-d2ca8e55cf09.png)

Describe the problem that you'd like solved

Can we lazily load each collection's page so a carousel doesn't make its query until a patron encounters or approaches that carousel? For example, when the LGBTQ collection page first loads, no carousels should load because no carousels are visible. As the patron scrolls down, carousels should load as they come into view.

Proposal & Constraints

Faster loading times would make for a better user experience.

Limiting the search queries would reduce load on our servers from calling carousels that are never seen, in the case of a happy patron who finds something they want to read and clicks away on the very first carousel they see. Carousels appear on pages that are not collections (such as the front page), but we've found it helpful to add tables of contents to collections pages, which makes it more likely a patron will only load one or two specific carousels before finding what they're looking for.

We also tend to add a lot of explanatory text and header images before the first carousel, so there's something for the patron to be reading before they expect to see carousel results. A patron might never scroll far enough to see a carousel at all, which would save us 34 search queries (and 680 search results) each time that happens.

Additional context

We're likely to keep creating large collections with many carousels on the same page because these collections are clear, easy to use, and I've received extremely positive feedback from patrons who enjoy the experience of browsing by subjects in the LGBTQ collection in particular. Even if they're not planning to read a book, several patrons had no idea this amount and variety of LGBTQ books even existed.

Huge collections are great for the curator (who gets to see the tangible, colorful results of their merging & tidying work) and great for patrons who want to browse something that feels vast but manageable.

Stakeholders

@libjenner

cdrini commented 2 months ago

@jimchamp recently implemented carousel caching on the server-side (#9807), which should greatly speed up these pages! That will be going out tomorrow to production, and we'll want to monitor it to see how the performance is doing. With any luck, it should make things fast enough that we won't need to implement lazy loading!

(eg https://testing.openlibrary.org/collections/LGBTQ )