search

arumoy-shome / UWNerdist

Got free time? Check out what classes are going on at the UW campus that interest you!
MIT License
1 stars 1 forks source link

Caching/Minimizing API calls #31

Open arumoy-shome opened 7 years ago

arumoy-shome commented 7 years ago

I wanted to get a discussion going on how we can avoid making multiple API calls per request, per user. Even though the UW API has no call limits, it's not the fastest and the limit will change in the future so it's in our best interests to limit the number of requests we make. Right now, there are no caching mechanisms in place, the app makes a new request to the API every time you visit the home page (schedules#index). Another reason why I definitely want to cache the results is because we need to grab the class schedule for all classes which I am guessing is a huge number. The API does not have an endpoint that will give us this data in a single request, so we need to make individual requests for each subject and retrieve it that way. Another thing to note is that the data will only change every 4 months (i.e. beginning of a new term). I have skimmed through the inbuilt caching mechanisms that Rails provides and I am aware of Redis however I feel that we will be better served if we persist the data rather than cache it. Thus for the first release, I intend to solve this problem in the following way:

  1. Introduce a persistence layer to save schedules for all subjects for a term
  2. Introduce a Job that runs every 4 months and updates the schedules to reflect the changes. At this point we can purge the entire db and only store the data for the current term or we can keep it all. Haven't given this much though so we will save that for later.

I would love to hear your thoughts, comments and concerns. 😊

arumoy-shome commented 7 years ago

This part of the app has to be pulled out into chron jobs that run at specific intervals (roughly every 4 months or so).