search

woocommerce / facebook-for-woocommerce

A first-party extension plugin built for WooCommerce. Development is managed by Ventures.
https://woocommerce.com/products/facebook/
GNU General Public License v2.0
211 stars 138 forks source link

Too many request scrapping #5

Closed DanielEvolufarma closed 7 years ago

DanielEvolufarma commented 7 years ago

After activating the plugin, my access logs shows a lot of hits like this

[31/Jul/2017:14:58:04 +0200] "GET /medicamentos/702845-nicorette-2-mg-chicles-medicamentosos-30-chicles HTTP/1.1" 301 3843 "-" "facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)"

I guess that's that facebook is webscrapping the e-commerce to get all the updated products.

Is that correct?

This is causing performance issues in the Mysql. Can I configure the scrapping interval? Can I configure any kind of soft-but-longuer scrapping? Can I configure the hour of the scrapping? Thanks

aashay commented 7 years ago

Daniel, we don't actually scrape anything in our WooCommerce plugin. What you see there is regular Facebook servers making sure your products are valid. Again, that's not coming from our plugin, that's just Facebook systems doing their due diligence.

I don't expect those requests to last, typically we only do that after initial upload and review.

On Mon, Jul 31, 2017 at 7:23 AM DanielEvolufarma notifications@github.com wrote:

After activating the plugin, my access logs shows a lot of hits like this

[31/Jul/2017:14:58:04 +0200] "GET /medicamentos/702845-nicorette-2-mg-chicles-medicamentosos-30-chicles HTTP/1.1" 301 3843 "-" "facebookexternalhit/1.1 (+ http://www.facebook.com/externalhit_uatext.php)"

I guess that's that facebook is webscrapping the e-commerce to get all the updated products.

Is that correct?

This is causing performance issues in the Mysql. Can I configure the scrapping interval? Can I configure any kind of soft-but-longuer scrapping? Can I configure the hour of the scrapping? Thanks

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/facebookincubator/facebook-for-woocommerce/issues/5, or mute the thread https://github.com/notifications/unsubscribe-auth/AAYLzA1qZeSiddrYuT75EsZukV25RbXXks5sTeNXgaJpZM4OodPa .