PHP Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 5242880 bytes) in /wordpress/core/5.7/wp-includes/class-wp-query.php on line 2986

kosiew commented 3 years ago

🔎 Isolate the bug

[x] I have confirmed this occurs in the most recent version of WordPress, WooCommerce, and Facebook for WooCommerce.
[x] I have confirmed this occurs when only WooCommerce and Facebook for WooCommerce are active and when using a default WordPress or WooCommerce theme.

✍️ Describe the bug

wp-admin fails

PHP Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 5242880 bytes) in /wordpress/core/5.7/wp-includes/class-wp-query.php on line 2986

🚶‍♀️ Steps to reproduce

It happened even with only these plugins:

akismet coblocks health-check jetpack woocommerce full-site-editing

✔️ Expected behavior

wp-admin loads without error

🗃 Logs

SSR uploaded in https://github.com/woocommerce/woocommerce-facebook/issues/8

WiredWonder commented 3 years ago

I too am getting out of memory issues, but when generating the feed.

danielbitzer commented 3 years ago

@kosiew we really need a full stack trace of this error to investigate? Could you please check your site's error logs for this?

WiredWonder commented 3 years ago

This is the error from the Apache error log. What other log would you like me to provide?

Cheers, Jason

danielbitzer commented 3 years ago

@galapogos01 nothing came through in that email. It might be that Github doesn't support attachments in replies via email.

kosiew commented 3 years ago

Hi, Here are the errors from facebook_for_woocommerce:

03-19-2021 @ 21:28:59 - There was an error trying sync products using the Catalog Batch API for job 24d592a27b5c1e0f05291a111b5f1e01: Cannot create the API instance because the access token is missing.
03-20-2021 @ 11:17:44 - There was an error trying sync products using the Catalog Batch API for job 5ceac627a90028678d4f54ba919149b2: Cannot create the API instance because the access token is missing.

and from fatal-errors around the same timestamp:

2021-03-19T02:18:00+00:00 CRITICAL Allowed memory size of 268435456 bytes exhausted (tried to allocate 16777224 bytes) in /wordpress/core/5.7/wp-includes/wp-db.php on line 2664

2021-03-19T03:48:36+00:00 CRITICAL Allowed memory size of 268435456 bytes exhausted (tried to allocate 16777224 bytes) in /wordpress/core/5.7/wp-includes/wp-db.php on line 2664

2021-03-19T06:24:57+00:00 CRITICAL Allowed memory size of 268435456 bytes exhausted (tried to allocate 16777224 bytes) in /wordpress/core/5.7/wp-includes/wp-db.php on line 2664

2021-03-19T08:00:04+00:00 CRITICAL Allowed memory size of 268435456 bytes exhausted (tried to allocate 16777224 bytes) in /wordpress/core/5.7/wp-includes/wp-db.php on line 2664

2021-03-19T10:37:06+00:00 CRITICAL Allowed memory size of 268435456 bytes exhausted (tried to allocate 16777224 bytes) in /wordpress/core/5.7/wp-includes/wp-db.php on line 2664

2021-03-19T21:32:03+00:00 CRITICAL Allowed memory size of 268435456 bytes exhausted (tried to allocate 8388616 bytes) in /wordpress/core/5.7/wp-includes/wp-db.php on line 2664

2021-03-19T21:32:18+00:00 CRITICAL Allowed memory size of 268435456 bytes exhausted (tried to allocate 8388616 bytes) in /wordpress/core/5.7/wp-includes/wp-db.php on line 2664

2021-03-19T22:54:44+00:00 CRITICAL Allowed memory size of 268435456 bytes exhausted (tried to allocate 20480 bytes) in /wordpress/core/5.7/wp-includes/wp-db.php on line 2022

WiredWonder commented 3 years ago

@danielbitzer I did not attach anything. The full apache log entry is in the topic of the ticket. If you need something else just let me know what it is you need.

danielbitzer commented 3 years ago

@kosiew how many products do you have on your site where you can reproduce the issue?

danielbitzer commented 3 years ago

Also @kosiew could you please try setting the wc_facebook_for_woocommerce_version option to 2.3.3 in your database. Then test if the problem goes away after that. This will tell us if it's the update routine that's causing the memory issue.

gamecreature commented 3 years ago

@danielbitzer I have the same problem. The script requires > 256 MB of memory to generate the Facebook feed. Total number of products: 1471. Total number of variants: 141299. (That last number is strange, I'm going to debug that, possible issue with WooCommerce update )

My workaround for now is to disable WP_CRON in wp-config

define('DISABLE_WP_CRON', true) ;

Change the generation to once a day

// Change Facebook for WooCommerce feed generation interval
function my_custom_facebook_feed_interval( $interval ) {
    return DAY_IN_SECONDS;
}
add_filter( 'wc_facebook_feed_generation_interval', 'my_custom_facebook_feed_interval', 10, 2 );

And add a cronjob for this user crontab -e . With increased memory limit for generating the feed.

*/10 * * * * cd /site/root && /usr/local/bin/php -d memory_limit=512M wp-cron.php event run --due-now  --quiet

DID some research:

And I found that I have got a LOT of orphaned variants.. (132.000) I will run a cleanup (https://remicorson.com/select-delete-woocommerce-orphaned-variations/) and check if that solves the memory issue.

WiredWonder commented 3 years ago

I have 0 orphaned variations and only have this OOM issue during feed generation (not in wp-admin). Not sure if we should be creating a separate ticket @danielbitzer ?

I'd love to know how to manually run the feed generation outside of the action scheduler so I can get better logging/insight as to what is causing the memory blowout.

In my google product feed I had to loop through products with WP_Query 500 at a time and run wp_cache_flush() in between each loop.

budzanowski commented 3 years ago

Hi @galapogos01 I have a quick question. What are your feed settings in your catalog? I am asking about feed settings described in https://www.facebook.com/business/help/2284463181837648?id=725943027795860

WiredWonder commented 3 years ago

@budzanowski Should all be default - I have been using the plugin for a long time and have not changed any settings on the FB side. I will note that I now get warnings for missing Google Categories.

omarNIU commented 3 years ago

We have the same issue. We have over 13,000 published products and whenever the Facebook for Woocommerce plugin is activated we are getting the Allowed memory size of 536870912 bytes exhausted (tried to allocate 20480 bytes) in wp-includes/wp-db.php on line 2022 fatal error. We have increased our memory limit to 512MB but we are still facing issues.

Isn't there a way that products are processed in batches? And have the batch amount as a setting?

We also noticed that in get_product_count you are fetching all the product but you are not setting the argument 'no_found_rows' => true therefore this is also taking too much memory since it will loop through all the results which have been fetched (for pagination purposes which you do not need).

The same goes to get_products_to_be_excluded if you are setting the 'posts_per_page' => -1

budzanowski commented 3 years ago

@omarNIU The synchronization process and memory issues are our current focus. We are actively developing a solution that should accommodate sites of all sizes. We hope to present a solution soon.

I have a question. Although the Allowed memory issue is present in your site logs does the site still operate normally (the customer-facing part at least and the products are synchronized correctly in FB Catalog? )

omarNIU commented 3 years ago

@omarNIU The synchronization process and memory issues are our current focus. We are actively developing a solution that should accommodate sites of all sizes. We hope to present a solution soon.

I have a question. Although the Allowed memory issue is present in your site logs does the site still operate normally (the customer-facing part at least and the products are synchronized correctly in FB Catalog? )

Hi please note that I have also provided suggestions which should help (but not solve the issue). No the site unfortunately is not operating normally (attaching some new relic results as well), sometimes the site is very slow.

WiredWonder commented 3 years ago

I just did a quick test against master which now includes the pull request 1840. The Action Scheduler task wc_facebook_regenerate_feed is still running out of memory on my site when started from wp-admin or by Action Scheduler.

PHP Fatal error: Allowed memory size of 367001600 bytes exhausted (tried to allocate 20480 bytes) in /wp-includes/meta.php on line 1078PHP message: PHP Fatal error: Allowed memory size of 367001600 bytes exhausted (tried to allocate 20480 bytes) in /wp-includes/class-wp-fatal-error-handler.php on line 72', referer: /wp-admin/tools.php?page=action-scheduler&action=-1&action2=-1&orderby=schedule&order=desc&status=pending

budzanowski commented 3 years ago

@galapogos01 Thanks for the test. I should have made a more direct statement in the PR about which parts of the code this affects, sorry for not being clear enough! The changes in the #1840 are not affecting feed sync. They are targeting Batch API sync. The plugin has both of those mechanisms of synchronization. Batch API is used if you click: We had a very non-optimal code that was initiating this procedure. Which should be fixed with #1840. The next step for us is to make the Batch API as performant as possible and further down the road start working on the feed file generation. How exactly the system will work depends on our discussions with Facebook.

WiredWonder commented 3 years ago

Hey @budzanowski sorry to bug you but do you have an update on the fix for feed file generation? I recently added a couple hundred more products and now the feed file generation AS job fails every single time even with 400MB process size. My products are drifting away from FB :D

haszari commented 3 years ago

do you have an update on the fix for feed file generation

No update or ETA on this sorry @galapogos01 .

I recently added a couple hundred more products and now the feed file generation AS job fails every single time even with 400MB process size

For the record, can you confirm a couple of details:

Total number of products on your site.
Versions of WooCommerce, Facebook for WooCommerce and anything else on your site. Easiest way is to upload a WooCommerce status report to this issue - this doc explains how to get your site's report.

Thank you 👍

WiredWonder commented 3 years ago

☹

4000 products
Currently Woocommerce 5.4.1, Facebook for Woocommerce 2.6.1. These have changed given the age of the defect, generally I keep up to date. Can provide other info on request (not publicly).

Happy to test any proposed changes to provide feedback.

woocommerce / facebook-for-woocommerce