Previously, our backend directly queried data from GraphQL then stored it into a hashmap, which sort of acted as a lite caching method to handling professor data so that we don't constantly query GraphQL (and piss off RMP). The flaw with this is if the backend server ever restarts, the hashmap's data is flushed.
These changes aims to fix that by storing all data in a MySQL database and grab data from there rather than solely the server. This should speed up the process of grabbing data significantly. Additionally, the professor's data is automatically re-queried if their data has exceeded 3 months since the last time their data was changed (i.e., first queried).
Furthermore, I changed the db table that handles professor names to use RateMyProfessors names instead of scraped names, which should guarantee all professors that actually have any ratings at all (this doesn't guarantee all professors will be able to be seen via the extension, but that would also mean they don't have any ratings).
A big issue to consider is dealing with professors that have duplicate entries in RateMyProfessors. Our current fix to that is to take the 1st entry and display that as data, but this can be very misleading (i.e., low review count/inflated rating). It works now though, so it's whatever.
Previously, our backend directly queried data from GraphQL then stored it into a hashmap, which sort of acted as a lite caching method to handling professor data so that we don't constantly query GraphQL (and piss off RMP). The flaw with this is if the backend server ever restarts, the hashmap's data is flushed.
These changes aims to fix that by storing all data in a MySQL database and grab data from there rather than solely the server. This should speed up the process of grabbing data significantly. Additionally, the professor's data is automatically re-queried if their data has exceeded 3 months since the last time their data was changed (i.e., first queried).
Furthermore, I changed the db table that handles professor names to use RateMyProfessors names instead of scraped names, which should guarantee all professors that actually have any ratings at all (this doesn't guarantee all professors will be able to be seen via the extension, but that would also mean they don't have any ratings).
A big issue to consider is dealing with professors that have duplicate entries in RateMyProfessors. Our current fix to that is to take the 1st entry and display that as data, but this can be very misleading (i.e., low review count/inflated rating). It works now though, so it's whatever.