Open adambechtold opened 8 months ago
It is easy to predict what kind of playlists users will request. For any combination of users, there are only 3 options.
▲ Pro - Achieves 🎯 Goal - playlists appear very quickly ▲ Pro - Partially achieves 🎯 Goal - time required is not directly proportional ▼ Con - Does not decrease the actual amount of time required to generate playlists; It just hides it from a user
taste-comparison
page, generate all 3 playlists and store them in a cacheAny time we load the taste-comparison
page, create all 3 playlists. When a user requests the actual playlist, just pull it from the cache.
▲ Pro - Easy to implement
▼ Con - Does not address the case when users navigate directly to a taste-comparison
with query parameters that specify a playlist
Run a cron job that creates all possible playlists once a day.
▼ Con - The size of the task would grow quickly ▼ Con - Does not address the experience for new users added that day
Right now, we store each event individually and perform aggregations (counts) for every playlist generation. We may be able to reduce the number of data points if we aggregate individual events into a time-series format.
▲ Pro - Decreases the time required to generate a playlist ▼ Con - Increase data storage ▼ Con - Requires a migration
This PR implemented the approach to Eagerly Generate Playlists and Store in a Cache
using the on-page-load variant.
This dramatically increased the speed.
This PR didn't decrease latency, but it makes the experience of waiting easier because it's clear something new is coming.
Background
The time it takes to generate a playlist is increasing as the amount of listening history we store grows.
Goals
Example
https://github.com/adambechtold/taste-explorer/assets/22563063/3015c9a7-4f14-4a9a-bbac-3e83573f67dd