search

FriendsOfPHP / Goutte

Goutte, a simple PHP Web Scraper
MIT License
9.26k stars 1.01k forks source link

How to bypass captchas and cloudfare protection? #450

Closed msouto346 closed 2 years ago

msouto346 commented 2 years ago

Hello, I am using this package and in local development it works great... but when on production with a non residencial IP my requests get blocked by cloudfare, for example:

"Attention Required! | Cloudflare body{margin:0;padding:0} if (!navigator.cookieEnabled)...

Any way to workaround this? I have googled quite a bit and found nothing that worked.

stof commented 2 years ago

Well, if the prod server is protected by Cloudflare's bot protection features, there is nothing you can do as it is legimitate for them to block your bot if you asked it to do it.

msouto346 commented 2 years ago

Well, if the prod server is protected by Cloudflare's bot protection features, there is nothing you can do as it is legimitate for them to block your bot if you asked it to do it.

What I am saying is the website I am scraping allows me to do it in my local development (residential ip) but when on production with my digital ocean ip it gets blocked. And I was checking if there is any possible workaround for it. I'm sure there is because I have seen other bots doing this exact same thing (check ps5 stocks from this website).

fabpot commented 2 years ago

Not related to Goutte anyway.