thinkshout / wp-otis

WordPress plugin for the Oregon Tourism Information System
GNU General Public License v2.0
4 stars 1 forks source link

Taste Newberg caching/async rework #17

Closed BrendanJercich closed 3 years ago

BrendanJercich commented 3 years ago

This update juggles the history import system a bit so that, if a call to the history API endpoint produces more than 500 results, the subsequent fetches are enqueued as jobs instead of recursive calls. I ran into an issue where I couldn't go back more than a month or so because the web host would kill off any jobs that lasted more than two minutes. This code should be more robust, if a little slower, and uses WP transients to store the merged set of updates/deletes between jobs.

jamacon36 commented 3 years ago

@BrendanAdkins this is amazing work! Testing things locally I'm seeing everything work up to the history import. The history import seems to have been stuck to page 1 only for the past hour or so: Screenshot 2021-08-25 at 10-36-10 Import Log ‹ Travel Oregon — WordPress

Is this expected behavior or might something be up?

If it's expected it might be worth updating the logged message on line 465 to manage expectations a bit.

Otherwise: seriously cool work 🙌

BrendanJercich commented 3 years ago

I took a deep look at this over the last couple of days and finally determined that the issue here is with the difference in Wordpress object caching between a server environment and our local Travel Oregon environments (where transients, and indeed WP's object caching system as a whole, don't really work without Redis). I'm going to write up a new issue to figure out how to make bulk imports work on developer environments, but outside of TO/Redis/Pantheon->local situations, the changes to the plugin should be fine as long as core Wordpress acts the way we expect it to.