search

downforacross / downforacross.com

Web frontend for downforacross.com -- continuation of stevenhao/crosswordsio
https://downforacrosscom.downforacross1.now.sh
MIT License
229 stars 90 forks source link

optimize puzzlelist: page geometrically instead of linearly #134

Closed stevenhao closed 3 years ago

stevenhao commented 3 years ago

Embarassingly, searching the puzzle repo accounts for almost all of the remaining hosting costs of downforacross [after migrating the game events to postgres]. this change should drastically reduce the amount of data being read from firebase.

the entire puzzle list is about 3MB, with ~10000 entries. The algorithm for search often behaves extremely inefficiently, because

  1. client-side filters the puzzle list items
  2. fetches the last 1 page, then last 2 pages, then last 3 pages, until it has enough post-filter results to render the page up to where the user has scrolled to.

In total this means that doing a single search for a keyword like "foobar" will cause the client to download the entire puzzle list 10000 / 40 = 250 times, for a total of 600MB data transferred.

This PR reduces that worst case behavior to ~6MB (100x) by doubling the number of pages loaded every time, resulting in the number of MB being downloaded in successive queries to form a geometric series with sum bounded by ~2x the size of the total repo.

image

vercel[bot] commented 3 years ago

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/downforacross/downforacrosscom/bysmd7ti3
✅ Preview: https://downforacrosscom-git-optimize-puzzle-list.downforacross1.vercel.app