oss-aspen / 8Knot

Dash app in development to serve open source community visualizations using GitHub data from Augur. Hosted app: https://eightknot.osci.io
MIT License
47 stars 59 forks source link

Move search option caching serverside #428

Closed JamesKunstle closed 12 months ago

JamesKunstle commented 1 year ago

Currently, the 'projects' that users pick from when using the search bar (which might be repos or orgs, if the user isn't logged in, or include user groups if the user is logged in) are cached in browser localStorage. Dash handles transporting this data to server-side-executing callbacks when needed.

If there are many options, however, this is expensive, especially considering that we refine the search for each character that the user types. That's a lot of I/O.

Likewise, if we cache user groups client-side, the same problem arises.

Instead, all option caching should be done server-side, including default options (for all anonymous users) and user options.

When an app instance boots, all of the options available by default will be cached in Redis. We can schedule a Celery Beat task to periodically update the options (maybe daily?).

When a user logs in, their groups will be cached in Redis as well. A pointer will be set in their server-side user session (some key in Redis) under which a blob of their options will be stored.

In this way, we can minify the responsibilities of the client application, the I/O required to use the search bar, and protect the user groups at once.

JamesKunstle commented 12 months ago

Currently, all options are server-side. User groups were clientside but those have been moved.

JamesKunstle commented 12 months ago

Should create a new issue for cache-backed options but this issue is done, 95%