magento facebook catalog feed stop fetching

[harendraprajapati]

https://magento.stackexchange.com/questions/230171/magento-facebook-catalog-feed-stop-fetching

when going to fetch products it stops after 10Mb creat file facebook_adstoolbox_product_feed.tsv Magento has around 42k products and its only fetched 16k and then stop running. I changed all limit in php.ini but still issue. also some change in facebook-for-magento/app/code/community/Facebook/AdsExtension/Model/FBProductFeed.php but still issue.

[dmitridr]

Hi,

Thanks for reporting.

Could I ask what is your version? Please upgrade to 2.5.0 if it is not 2.5.0, older versions are slower and had performance and memory issues.

Could you share the URL of your server or your store ID? We have some logging on the FB side which we can potentially use to determine what went wrong.

There should be some files named facebook_* in your magento log directory that logs the progress of the feed file creation. The default log directory is <magento_dir>/var/log/ , if you have logging enabled. If you don't have logging enabled, please enable logging and try another sync. Could you send us these files via email to ads_extension@fb.com ?

With this information we should be able to at least pinpoint the problem. I suspect it's OOM or PHP time limit killing the feed generation.

[harendraprajapati]

Yes, I am using the latest version 2.5.0. I shared details on mail. Please check let me know.

[dmitridr]

Great, I got your email, looking at it now, will update soon.

[dmitridr]

It looks like after ~200s the process generating the feed is being terminated. This behavior is pretty consistent in the log file.

You can try making this change in FBProductFeed.php to verify : try adding a sleep statement, sleep(201), and doing a feed fetch via the Fetch Now button.

(1) If it uploads nothing, it's definitely being terminated. (2) If it still uploads 16k it's being caused by a product which is configured in an unexpected way.

I have looked at many of these log files for both (1) and (2), and I'm pretty sure yours is (1), though I would still try adding a sleep and testing it out to be 100% sure.

To solve the timeout, I would try to locate the php timeout settings on your server itself, there may be multiple places that may need to be changed, not just php.ini.

[harendraprajapati]

I try adding a sleep statement, But Still the same issue. still uploads around 16k it's being caused. Every time fetching size around 10mb /media/facebook_adstoolbox_product_feed.tsv

[dmitridr]

Strange. Where have you added the sleep? I would be sure to add it on the critical execution path, for example here.

If it still doesn't work I would ask you to send us your other (non facebook) error logs as well, if any appear. There may be an exception there that we can look at. The section of the logs during the time of the feed generation would be enough.

One question, when you retry is the last product in the .tsv file always the same, or different?

[harendraprajapati]

Added the sleep as you said location. and also added a top. But still, issue. product in the .tsv files always the different.

[dmitridr]

Very strange, it appears we ruled out both possible causes. If the last product is different, it is not crashing because of a specific product.

I think maybe sleep does not count towards the total execution time of the script, so this was a bad way to test that the problem is timeout, so we can't rule it out completely. If you'd like to try another test, I suggest inserting while(1){} into the code in the same spot earlier, and see if the php process vanishes after ~200s. (You may have to kill the process manually if you do this and the script doesn't die).

Here's some more suggestions for you to try :

(1) Check to see you have more than enough disk space to write the feed file. This could also break the feed generation. (2) Check to see that your memory limit is high (at least 512MB) for PHP. Use top or another tool to monitor the memory usage of the feed generation and see if it is approaching close to the limit (the logs say it's not but this method is more accurate). (3) Make sure you are using the Fetch Now inside the popup created by the plugin. Other testing methods (such as refetching the feed using the catalog manager interface) may hit the cached feed file instead of generating the feed.

[harendraprajapati]

I don't understand what is going on last 4 days it is working fine. But Today again the same issue. you see my store details.

[dmitridr]

We had another issue like this in the past which only happened sometimes, #12, maybe check to see if this sounds similar to your case? This was probably caused by a memory leak, as far as I could tell, so trying to increase memory may help.

Only other thing I can think of is that the DB was overloaded during the feed generation process, which caused a failure in the generation. If that is the cause you can try running the feed generation during times which your server is not as busy, or adding sleep statements during generation, this may help.

Another thing you can try is changing the batch maximum here, this may improve/decrease performance based on your server's settings.

Again, I would ask you to consider sending us your (non facebook) error logs, especially if you see any entry related to facebook in these logs. There could definitely be a hint in there towards the problem.

[harendraprajapati]

Still Facing Issue. Server Support said not our issue.

[dmitridr]

Unfortunately we cannot reproduce it. So it is likely a combination of server setting and plugin code that is causing the problem.

That you said it worked fine for 4 days and then not leads me to believe it is some very strange race condition on your server. Also this is not affecting everyone, but there are some few others reporting similar problems, see for example #102 .