JimmyLaurent / cloudflare-scraper

A package to bypass Cloudflare's protection
MIT License
284 stars 30 forks source link

why looking for "Just a moment..." in title instead of checking if the `cf-browser-verification` tag in the HTML code is present? #2

Closed unixfox closed 4 years ago

unixfox commented 4 years ago

Cloudflare allows to change some part of the template of the verification page. I think looking for the words "Just a moment..." in the title is not a good idea because a website could bypass your program just by changing the title of the verification page: https://github.com/JimmyLaurent/cloudflare-scraper/blob/master/src/getCookies.js#L24 It's better to look for cf-browser-verification or another "generic" tag in the HTML code because that is something that the owner can't change because it's part of the "verification" JavaScript code of Cloudflare:

let pageContent = await page.content();
while (pageContent.includes("cf-browser-verification")) {
    response = await page.waitForNavigation({
      timeout: 45000,
      waitUntil: 'domcontentloaded'
    });
    pageContent = await page.content();
    count++;
    if (count === 10) {
      throw new Error('timeout on just a moment');
    }
  }
JimmyLaurent commented 4 years ago

At this early stage, it's more like POC than a real solution. I also used request, which is a deprecated package. But I'm open to all suggestions and this is a good one, thanks, I'll make the change. Don't hesitate if you have more.

JimmyLaurent commented 4 years ago

I changed it.