Scraping a site with CloudFlare protection/redirect returns no results

spekulatius / PHPScraper

A universal web-util for PHP.

https://phpscraper.de

GNU General Public License v3.0

509 stars 73 forks source link

Scraping a site with CloudFlare protection/redirect returns no results #203

Closed gelform closed 6 months ago

gelform commented 6 months ago

When scraping a url that has a CloudFlare firewall, the "Please wait while your request is being verified..." javascript redirect is returned instead of the HTML.

Current examples: $web->go('https://eynzone.com') $web->go('https://brilliantly.net')

Running the code on my local environment works, but not on my server.

Any known ways to get around this?

spekulatius commented 6 months ago

Hey @gelform

I guess this is due to the IP of the webserver. I haven't got any way around this.

Let me know if you identify a way to solve this.

Cheers!

gelform commented 6 months ago

Makes sense. Thank you so much for getting back to me! And thank you for such a great library.