Due to the update to BC Ferries site, the scraper now has to scrape several urls instead of a single one so it takes much longer. The current setup is the scraper runs every 3 minutes, and this worked when it was quick but now there's a period of time after each update where the api can take 3-5 seconds to return a value, instead of the usual 100-200 milliseconds.
Not sure the best solution to this, if anyone knows a good way to separate them in such a way that updating the db doesn't slow down the api please comment it on this issue.
Due to the update to BC Ferries site, the scraper now has to scrape several urls instead of a single one so it takes much longer. The current setup is the scraper runs every 3 minutes, and this worked when it was quick but now there's a period of time after each update where the api can take 3-5 seconds to return a value, instead of the usual 100-200 milliseconds.
Not sure the best solution to this, if anyone knows a good way to separate them in such a way that updating the db doesn't slow down the api please comment it on this issue.