hardkoded / puppeteer-sharp

Headless Chrome .NET API
https://www.puppeteersharp.com
MIT License
3.24k stars 431 forks source link

Redirection from Cloudflare Waiting Room is blocked #2426

Open starkdm opened 5 months ago

starkdm commented 5 months ago

The site I'm trying to explore with Puppeteer is protected by a Cloudflare page that analyzes the browser. It appears that there is some problem when using GoToAsync - the Cloudflare waiting room is endlessly updating, but does not block access. As soon as I close the application (= close the CDP session), the redirection is immediately completed successfully. If I open the tab manually and type in the URL, the check is also successful. Either Cloudflare somehow understands that the browser is automated, or there is a problem in Puppeteer. How can we understand this?

In addition, if you create a new tab and load the page, then continue execution from the line IPage page = await PuppeteerHeadful.NewPageAsync();, the check will be successful.

await new BrowserFetcher().DownloadAsync();
PuppeteerHeadful = await Puppeteer.LaunchAsync(new LaunchOptions {
    Headless = false,
    Devtools = false,
    Browser = SupportedBrowser.Chrome,
    DefaultViewport = null,
    Args = new[] {
        "--disable-blink-features=AutomationControlled"
    }
});

IPage page = await PuppeteerHeadful.NewPageAsync();
await page.GoToAsync(url);

URL for example: https://www.vindecoderz.com/EN/check-lookup/WVWMP7AN2BE710202 .NET version: 6.0 PuppeteerSharp version: 14.0.0

kblok commented 5 months ago

There are some conversations upstream about that.

starkdm commented 5 months ago

@kblok Found this discussion earlier. I think I have a different problem because I don't see any warnings from Cloudflare.