search

me-next / menext-backend

**THIS IS DEPRECATED** The server for MeNext, an app for collaborative Spotify DJing
1 stars 0 forks source link

Metadata Caching #35

Open daniel-centore opened 7 years ago

daniel-centore commented 7 years ago

Cache metadata on the server so we don't bump up again spotify's rate limiting with many users.

The metadata should be removed from cache when it gets stale to keep memory usage low and so we get metatdata updates if it ever changes (which should be infrequent but can happen).

ConnorFoody commented 7 years ago

Do we want to make a separate service for this to avoid duplicates?

Maybe have @mluquette look into this?

daniel-centore commented 7 years ago

Do we want to make a separate service for this to avoid duplicates?

What do you mean by "separate service"? Like an entirely separate program?

What do you mean by "avoid duplicates"? What duplicates are we avoiding?

I personally pictured this as being part of the same program, but otherwise completely separate endpoints and no code sharing (other than server setup and endpoint routing).

Maybe have @mluquette look into this?

Go for it. The way I picture it working is with endpoints for {track, artist, album} which accept a list of Spotify IDs and return the metadata for those URIs, either from our personal cache or by asking Spotify for them.

Do we want to think about setting up a database for this?

Quickstart: https://developer.spotify.com/web-api/endpoint-reference/

ConnorFoody commented 7 years ago

I thikn we are generally on the same page with design here, the only difference is combining the cache with the menext platform or not.

I would go with a separate service because I don't like platforms. I have found that platforms are harder to develop and less flexible to deploy. It creates a single point of failure and forces you to scale them at the same time. Additionally this is a natural fit for a separate service since there is no shared code.

A good DB for this might be Memcached or Redis. If you're serious about scaling this you should move to a real cloud provider like AWS. The clould platform should provide some managed caching services.

daniel-centore commented 7 years ago

I would go with a separate service

Additionally this is a natural fit for a separate service since there is no shared code.

Alright, you make good points.

If you're serious about scaling this you should move to a real cloud provider like AWS.

Yeah, I might need to bite the bullet on that one. I have no time to set that up right now, though. For now we can abstract that away and do in memory with a key-value store in a separate class. It would be easy to switch out the backend later, then.