Trash Managment Loop In History Import

[jamacon36]

Overview

This adds an array to populate with post ids that have the verb delete during the _import_history function on 496 of Otis_Importer. This is then looped over after the updates are made to relevant posts and trashes the deleted posts.

Testing

• [x] Apply this PR to a testing environment
• [x] Wait for a history import to run
• [x] Check that deleted POIs have been deleted in WP

[BrendanJercich]

This makes sense to me as a strategy--did you actually find examples where updates happened after deletes?

[BrendanJercich]

Hmmmm, unfortunately here's what happens in the Scheduled Actions log:

2021-10-21 23:18:39 +0000
action created
2021-10-21 23:19:01 +0000
action started via Async Request
2021-10-21 23:19:22 +0000
unexpected shutdown: PHP Fatal error Uncaught ArgumentCountError: Too few arguments to function array_flatten(), 1 passed in /code/web/wp-content/plugins/wp-otis/src/Otis_Importer.php on line 1023 and exactly 2 expected in /code/vendor/rappasoft/laravel-helpers/src/arrays.php:233 Stack trace: #0 /code/web/wp-content/plugins/wp-otis/src/Otis_Importer.php(1023): array_flatten(Array) #1 /code/web/wp-content/plugins/wp-otis/src/Otis_Importer.php(788): Otis_Importer->_translate_taxonomy_value('glocats', Array) #2 /code/web/wp-content/plugins/wp-otis/src/Otis_Importer.php(401): Otis_Importer->_upsert_poi(5580596, Array, false) #3 /code/web/wp-content/plugins/wp-otis/src/Otis_Importer.php(136): Otis_Importer->_import_pois(Array) #4 /code/web/wp-content/plugins/wp-otis/src/Otis_Dashboard.php(39): Otis_Importer->import(Array, Array) #5 /code/web/wp/wp-includes/class-wp-hook.php(287): Otis_Dashboard->otis_start_import(Array, Array) #6 /code/web/wp/wp-includes/class-wp-hook.php(311): WP_Hook->apply_filters('', Array) #7 /code/web/wp/wp-includes/plugin.php( in /code/vendor/rappasoft/laravel-helpers/src/arrays.php on line 233

[BrendanJercich]

(I'm testing this out on develop-tror)

[BrendanJercich]

Weird and annoying that this appears to be related to glocats and how those get translated to WP taxonomies, somehow, nothing directly related to the code changes.

[BrendanJercich]

Changing INF to PHP_INT_MAX makes the scheduled action actually run! Waiting to see if a full-year import removes our rogue POIs now.

[BrendanJercich]

Well, this is unexpected. I ran a bulk update going all the way back to last October 1, and the first time, it hit a max process expiration error and gave up on a high page number:

Which was annoying but not unprecedented. I went back to run it again today and saw this at the beginning of the import process:

And this at the end:

Which made me think at first that the "page" parameter was not getting reset between runs and that was causing issues, but, no, the "history-page" param is still at 1 as expected. Doesn't make any sense that there would be 0 updates, though.

[BrendanJercich]

Tried adding some logging and running an import just back to the beginning of September. This definitely isn't right.

[BrendanJercich]

Is it possible something related to transients broke in the WP or PHP update?

[BrendanJercich]

Moved this work to the otis multidev, and the new logging confirms that the delete loop is getting hit, at least when I import starting with 2021-10-01.

[BrendanJercich]

That delete process also definitely removes the Flourish Mercantile and "Test" POIs that Cecilia listed here: https://3.basecamp.com/3337340/buckets/676097/todos/4239118628#__recording_4286408441 so there's some confirmation that this work is at least moving in the right direction.

[BrendanJercich]

And now I can confirm that if you start a history import with multiple pages, it gets stuck on the first page, apparently forever?

[BrendanJercich]

Nope, I lied! It's just the logging that is bad, it does hit the delete loop eventually.

[BrendanJercich]

The import shown above hit post 6098290, trashed it, and then eventually timed out--I suspect the job is not getting the "done" signal after the delete loop is complete.