Need a way to process existing posts in bulk

[binarygary]

Since we tag each post with post_meta _is_video I'm thinking maybe a simple cron task that would grab a handful of posts without _video meta?

[tw2113]

Since we have existing users already, I would personally advise against having anything like this auto-run without them knowing/manually triggering. However, setting up an action that runs it all would be an interesting idea. Perhaps from the WP LIst Page view of the posts/pages/post types. We could add our own button at the top that would collect the selected posts, and run it through.

[binarygary]

related: https://github.com/WebDevStudios/Automatic-Featured-Images-from-Videos/issues/7

[binarygary]

I think we need to be careful how we approach this batching.
You are right...it's conceivable current users could have thousands of posts...so a cron job would be required.

Up until now, we haven't had a settings page for this. Maybe we add a settings page that allows users to configure how much of the_content the plugin looks at? (defaulting to 800) As well as provides a button for batch updating all posts.

Your idea of adding a button on list page is pretty slick. Conceivably the user could filter down to CPT>of Taxonomy "Video" and select all of those. Once passed in we could treat it as a list of IDs to act on however we opt to do the batching.

[tw2113]

I'd dare say cron jobs are overkill for this. It's not something that's going to need to run regularly. It would be beneficial to have a way to provide a bulk value to run once, or smaller bulks to run as many times as needed to get them all done once. Beyond that, all the new posts coming in would already be covered.

[binarygary]

I don't necessarily mean a repeating cron job.
But in essence a user that has 4 bazillion posts could install this and then click "update all" posts while running on some kind of awful shared hosting...and it wouldn't work. Also, I wonder if youtube/vimeo would see that as some sort of bad thing (they may not care).

So, my thought is that it creates a task that grabs 10 posts and parses them.
If, after parsing those it finds 10 more posts without _is_video meta it schedules another instance of itself to run X minutes later.
Instead of being a regular interval task (cron) it just spawns itself through WP scheduling.

So, on the final run, it can't find any post without _is_video meta. It simply just doesn't schedule itself to run again.

So "...smaller bulks to run as many times as needed..." is that kinda what you were thinking?

[tw2113]

Gotcha. My biggest concern was it running for forever after having processed all that lacked. If we indeed do this, perhaps we could set it up to stop rescheduling if no non-covered posts exist any more.

On the flipside, and I think you agree with this sentiment overall, I still would prefer it not happen without user approval. Would also prefer post type control, so that it's not doing it for all possible ones. Perhaps at most, "post" post type by default, perhaps "page" too. We will want to avoid all non-public ones, logging post types come to mind, as well as any ecommerce based ones.

[binarygary]

Cool. We're very much in agreement on both points.
The logic on bulk-processing will be: Select X posts to process. (a query looking for posts that don't have _is_video set to T or F). Process those posts. Check if there are any more posts without _is_video. If no more posts are found, we're done. If not...then schedule another batch run.

What if we added a button to the top of posts list screen that says something like "check for video content" or something? https://www.dropbox.com/s/fix15gfj2k4wtjx/Screenshot%202016-11-19%2015.15.23.png?dl=0

[binarygary]

Alrighty, I pulled something together. Can you test functionality on this branch? https://github.com/WebDevStudios/Automatic-Featured-Images-from-Videos/tree/feature/bulk-update

[binarygary]

Two things I realized after sleeping on this. First, I'm not sure if the processing button will show on "pages" if "posts" are processing. Need to test that. Second, what should happen to the button after processing is done? Perhaps the button should check if we have any to process and if not, not display at all?

Finally, I opted not to refactor as I had some time constraints and thought it best to get the basic structure of this feature finished. This should be refactored to OOP.

[binarygary]

@tw2113 Will probably work on the js component tomorrow evening.
Would like some comments regarding: Second, what should happen to the button after processing is done? Perhaps the button should check if we have any to process and if not, not display at all?

[tw2113]

Perhaps hide once completely done and processed? and also not show at all if none to process.

[binarygary]

This last feature is now part of the logic in the ajax/js https://github.com/WebDevStudios/Automatic-Featured-Images-from-Videos/tree/feature/bulk-update

[tw2113]

Going to close as done for the moment. We can revisit in future bug fixes if needed. Planning to note that the bulk edit has potential for some bugs/issues and that it's a work in progress.