Closed BrendanJercich closed 3 years ago
@BrendanAdkins this is amazing work! Testing things locally I'm seeing everything work up to the history import. The history import seems to have been stuck to page 1 only for the past hour or so:
Is this expected behavior or might something be up?
If it's expected it might be worth updating the logged message on line 465 to manage expectations a bit.
Otherwise: seriously cool work 🙌
I took a deep look at this over the last couple of days and finally determined that the issue here is with the difference in Wordpress object caching between a server environment and our local Travel Oregon environments (where transients, and indeed WP's object caching system as a whole, don't really work without Redis). I'm going to write up a new issue to figure out how to make bulk imports work on developer environments, but outside of TO/Redis/Pantheon->local situations, the changes to the plugin should be fine as long as core Wordpress acts the way we expect it to.
This update juggles the history import system a bit so that, if a call to the history API endpoint produces more than 500 results, the subsequent fetches are enqueued as jobs instead of recursive calls. I ran into an issue where I couldn't go back more than a month or so because the web host would kill off any jobs that lasted more than two minutes. This code should be more robust, if a little slower, and uses WP transients to store the merged set of updates/deletes between jobs.