Open adambechtold opened 8 months ago
1) Create database 2) Create api key to last.fm and spotify 3) Add users using admin api 4) Run cron jobs to backfill listening history
â–¼ Con - Requires collaborators to have their own last.fm account and spotify developer account â–¼ Con - Complicated and time-consuming
Create a very small database on AWS with example data and make read-only permissions to it public.
â–² Pro - Very easy for collaborators to get started â–¼ Con - Performance could degrade as more people use it â–¼ Con - Cost - This isn't free
â–² Pro - Free â–¼ Con - Use by collaborators could degrade prod's performance
Provide .dump
files with example data.
â–² Pro - Collaborators can choose their favorite database platform/approach â–² Pro - Easy to provide various dataset sizes
â–² Pro - Version control â–¼ Con - Some of the dumps could be very large, making the repo unnecessarily large
â–² Pro - Dumps can be very large â–¼ Con - Version control is harder
Collaborators could simply pull down this repo, spin it up, and start going.
â–² Pro - Very easy to get started
â–¼ Con - More work to create than the .dump
option
.dump
during start upHost a .dump
file on S3. Host a container on AWS container registry. Give the container an entrypoint script that checks if the database is already populated and, if not, downloads the dump and populates the database.
(See chat)
.dump
files are available on S3I was able to create dump files and put them on S3 but am running into some issues creating a docker container that can download them and pre-populate the database.
I'll keep working on it, but here are the .dump files in the meantime:
cc: @tusharwebd
Background
We're opening this project up to more collaborators.
Frictions
Goals