Problems with re-indexing large index

WebDevStudios / wp-search-with-algolia

Improve search on your site. Autocomplete is included, along with full control over look, feel and relevance.

https://wordpress.org/plugins/wp-search-with-algolia/

146 stars 55 forks source link

Problems with re-indexing large index #321

Closed timbolimboslice closed 1 year ago

timbolimboslice commented 1 year ago

Hi there,

I made some changes that required re-indexing a large index (~37,000 records). I've tried using the "Re-index" action and also the CLI command, however, the process gets to a certain point and the query is "killed". It always seems to fail around the same point (batch 83-84).

Things I've tried:

Starting from that batch (with --from_batch=84)
Adjusting the batch size.
Asked our hosting provider made some changes to allow long running queries.

We're on WordPress version 6.1.1 and on version 2.5.0 of the plugin.

Do you have any suggestions on how to get past this hurdle?

Thanks, Tim

Screenshot 2023-04-27 at 12 14 21 PM

tw2113 commented 1 year ago

What batch size did you adjust things to and in what method did you change that with?

Curious if you could identify exactly when it fails, and if it's always at the exact same post, or at least near it, and then do the math to get right before that and see if it fails pretty quickly and well before the batch is fulfilled for its target amount

timbolimboslice commented 1 year ago

Hey Michael,

After doing some debugging locally, I was able to determine that the issue isn't with the plugin at all. The process was being hung up by the the_content filter being ran on some old posts. A gallery shortcode without an ids attribute seems to the culprit.

I'll close this issue out, but hopefully this is useful to somebody in the future. Appreciate the reply.

tw2113 commented 1 year ago

Interesting, and good sleuthing there Tim