wc_facebook_regenerate_feed Out of Memory issues since 2.3.2

WiredWonder commented 3 years ago

🔎 Isolate the bug

[X] I have confirmed this occurs in the most recent version of WordPress, WooCommerce, and Facebook for WooCommerce.
[x] I have confirmed this occurs when only WooCommerce and Facebook for WooCommerce are active and when using a default WordPress or WooCommerce theme.

✍️ Describe the bug

Hi guys,

I have a store with over 3500 products.

Since upgrading to 2.3.2, every time the action scheduler task wc_facebook_regenerate_feed runs it is failing due to out of memory issues: unexpected shutdown: PHP Fatal error Allowed memory size of 419430400 bytes exhausted (tried to allocate 20480 bytes) in /wp-includes/functions.php on line 624

This happens every 15 mins and I have tens of thousands of failed tasks.

As you can see I have already had to raise the single thread memory limit to 400MB which is very high. I had to do this for prior versions to work at all, but it's been failing consistently since the version upgrade.

Removing the Google Product Categories from the config allows the cron instance of the task to run but the action scheduler still reports errors constantly, and Facebook complains that my products have less visibility due to not having Google categories.

Creating this as https://wordpress.org/support/topic/out-of-memory-issues-since-2-3-2/#post-14300819 led me here.

Thanks, Jason

🚶‍♀️ Steps to reproduce

Install the plugin on a large (3000+ product) site
Watch action scheduler task logs

✔️ Expected behavior

Scheduled tasks should not cause OOM errors.

🗃 Logs

Can provide specific logs on request

danielbitzer commented 3 years ago

Hi @galapogos01, thanks for the report!

Could you confirm this issue is present when using the latest (2.3.5) version of the plugin?

WiredWonder commented 3 years ago

Yes @danielbitzer . I think this is different to https://github.com/woocommerce/facebook-for-woocommerce/issues/1818 as 1818 relates to wp-admin and my ticket is the background AS task.

danielbitzer commented 3 years ago

Ok I see. We are currently working on improving the feed and product sync areas of the plugin.

@budzanowski since you've been looking into the feed is there anything we can do here. Sounds like that proposed option to disable the feed would be ideal.

WiredWonder commented 3 years ago

Confirming these are occurring worse in 2.4.0. Happy to provide any logs requested.

danielbitzer commented 3 years ago

@galapogos01 we'll be working on re-building the feed for the next release. 2.4 didn't make any changes to the feed specifically.

WiredWonder commented 3 years ago

Hey guys, how is this tracking? Still dealing with failures and high memory usage on this, reported months ago and hanging out for a fix!

jgreys commented 3 years ago

Hello! Same here. Wasn't having issues before the updates. (~1000 products per site) Increased the timeout, concurrent batches, memory limit... But still no luck. :(

danielbitzer commented 3 years ago

Howdy, work on this has unfortunately been paused for the moment. We're not sure when it will resume.

rafal-sokolowski commented 3 years ago

@danielbitzer Is there any workaround, quick fix or temporary solution we could apply in order to make it work? For the moment, the plugin seems to be useless.

WiredWonder commented 3 years ago

It's good at making your web server run out of memory :-(

rafal-sokolowski commented 3 years ago

It seems to work fine when I run it from CLI.

wp eval "(new SkyVerge\WooCommerce\Facebook\Products\Feed)->regenerate_feed();"

budzanowski commented 3 years ago

I understand the frustration. We have started work on a new feed generator, that should not experience memory issues. The work is at a fairly advanced stage. Unfortunately, we had to stop. I hope that we will restart the work soon and deliver memory issues free feed generator. PR for reference: https://github.com/woocommerce/facebook-for-woocommerce/pull/1989

WiredWonder commented 2 years ago

Hey @budzanowski thanks for the work on the PR / new feed generator. Do you know in which version this will be released? Can the bug be closed if I haven't verified it yet?

budzanowski commented 2 years ago

@galapogos01 Hey!

Can the bug be closed if I haven't verified it yet?

This is a GitHub feature. The issue was auto-closed when the PR got merged in master. It would be better to verify and then close the issue but most of the time we would end up with issues people forget to close :)

Do you know in which version this will be released?

The next one: 2.6.6. Early next week most probably. If you will find that this did not help then you can reopen the issue with a comment so we can follow up. Or open another Issue if you would find some other problem related to the new code. This is the flow that we use in GitHub for these things. Hopefully, it will work without problems.

jgreys commented 2 years ago

@budzanowski Thanks for the work! Looking forward to the upcoming update :)

WiredWonder commented 2 years ago

Legendary, thanks @budzanowski !!

rafal-sokolowski commented 2 years ago

Unfortunately, in my case, the update hasn't solved the issue.

2021/11/08 11:21:47 [error] 34576#34576: *4660912 FastCGI sent in stderr: "PHP message: PHP Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 20480 bytes) in /www/playground/public/wp-content/plugins/taxonomy-terms-order/include/addons.php on line 47" while reading response header from upstream, client: 34.194.65.22, server: playground.test, request: "POST /wp-admin/admin-ajax.php?action=as_async_request_queue_runner&nonce=1ab2e2b938 HTTP/1.0", upstream: "fastcgi://unix:/var/run/php8.0-fpm-z3tshop.sock:", host: "playground.test", referrer: "https://playground.test/wp-admin/admin-ajax.php?action=as_async_request_queue_runner&nonce=1ab2e2b938"

budzanowski commented 2 years ago

@rafal-sokolowski That looks like a memory issue that is comming from another plugin: /www/playground/public/wp-content/plugins/taxonomy-terms-order and not Facebook For WooCommerce.

rafal-sokolowski commented 2 years ago

@budzanowski That's true. I know how it looks like but I also got other entries in my log file. When it comes to error location, they are pretty random:

2021/11/08 11:21:47 [error] 34576#34576: *4660912 FastCGI sent in stderr: "PHP message: PHP Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 40960 bytes) in /www/playground/public/wp-includes/option.php on line 502" while reading upstream, client: 34.194.65.22, server: playground.test, request: "POST /wp-admin/admin-ajax.php?action=as_async_request_queue_runner&nonce=1ab2e2b938 HTTP/1.0", upstream: "fastcgi://unix:/var/run/php8.0-fpm-z3tshop.sock:", host: "playground.test", referrer: "https://playground.test/wp-admin/admin-ajax.php?action=as_async_request_queue_runner&nonce=1ab2e2b938"

What they have in common is admin-ajax.php?action=as_async_request_queue_runner and when I comment out schedule_feed_generation, no errors are showing up.

WiredWonder commented 2 years ago

@budzanowski the OOM error is generated by whatever thing next tries to get memory, usually in the main loop of whatever repetitive operation is running.

@rafal-sokolowski did you enable the new feed option? it's not on by default.

I have a 4500 item / 15k post site and have had no errors since enabling. Very happy @budzanowski !!!

budzanowski commented 2 years ago

@galapogos01 I am glad to hear that!

@rafal-sokolowski the pointer from @galapogos01 is correct, you need to enable the new generator first. More info in https://woocommerce.com/document/facebook-for-woocommerce/#section-25

rafal-sokolowski commented 2 years ago

@galapogos01 @budzanowski The option wasn't checked indeed. That's probably the case. Thanks for your help!

budzanowski commented 2 years ago

If you like the update and the progress the plugin is making then please leave a review at https://wordpress.org/plugins/facebook-for-woocommerce/ . We are trying to address a lot of plugin problems and build a solid and helpful product. We still have a long way to go for sure, but we are heading in the right direction.

Thank you all for participating in the process of making the plugin better with each iteration!

woocommerce / facebook-for-woocommerce