Open nrb opened 9 years ago
For the vendors, we will need to cache based on the vendor name, not just the 'bounty' function name.
Another thing to consider - caching these values for too long will cause stale values to exist in the cache. So, the weekly information could be cached for a maximum of 6 days, but only that cache isn't set right before the applicable reset.
For example, caching the nightfall for a day or more on a Monday will mean the cache value will not match after the weekly reset.
A more reasonable thing might be to just cache for some number of hours, like 6 or so.
Since this is going to be in a self-contained module with caching specifically for the bot, it could try to use some date math smarts to fudge dates/times on the border, but I'm not sure that's really messy.
One final thing that might be nice to have - appending a message when a response is first cached vs when it's pulled from cache.
If we switch to pulling data from bungie.net (#11) then this should probably be simplified as a single daily cache mechanic (all daily events and weekly events would be pulled down and parsed from a single payload).
True, though I still worry about when it would happen. I'm leaning towards using some kind of scheduled mechanism within helga (the bot this is based on) so that this could happen each day, rather than a per-request basis.
Yep. A daily scheduled "continuous cache" makes sense.
Owen Nelson
On Tue, Apr 28, 2015 at 10:52 AM, Nolan Brubaker notifications@github.com wrote:
True, though I still worry about when it would happen. I'm leaning towards using some kind of scheduled mechanism within helga (the bot this is based on) so that this could happen each day, rather than a per-request basis.
— Reply to this email directly or view it on GitHub https://github.com/nrb/destiny-info-bot/issues/5#issuecomment-97152377.
I think that we can actually use the response from Bungie's API to tell us this, which allows us to see what bungie's reset dates/times are.
Using these fields ('nightfallResetDate', 'heroicStrikeResetDate', 'dailyChapterResetDate', 'dailyCrucibleResetDate'), we should be able to make more efficient caching.
Since we'd be fetching information daily that contains the weekly information as well, it's likely easiest to update everything on a daily basis, possibly at the first request after the expiry. Also, since we have a consolidated request for most of this info now, only the API response would need to be cached, further simplifying the code base.
The weekly nightfall and heroic information could be cached for a period of 6 days. This would reduce the dependence on external calls and blocking the bot.
Similarly, the daily and bounty info could be cached for a shorter period of time.
This all should probably go in a new module that wraps the scraper calls, and is consumed by the
destiny_bot
functions.