search

berstend / puppeteer-extra

💯 Teach puppeteer new tricks through plugins.
https://extra.community
MIT License
6.38k stars 738 forks source link

webdriver detected #384

Closed riprekt closed 3 years ago

riprekt commented 3 years ago

Seems like webdriver is now being detected by different checks. (https://intoli.com/blog/not-possible-to-block-chrome-headless/chrome-headless-test.html for example).

I tried both stealth and methods of my own, but both get detected.

Niek commented 3 years ago

What version of puppeteer-extra-plugin-stealth are you running? There's a fix in 2.6.5 for this that adds --disable-blink-features=AutomationControlled. Can you check your version and post the output of chrome://version?

riprekt commented 3 years ago

+-- puppeteer-extra@3.1.15 | +-- @types/debug@4.1.5 | +-- @types/puppeteer@5.4.0 | | -- @types/node@14.14.10 | +-- debug@4.3.1 | |-- ms@2.1.2 | -- deepmerge@4.2.2 -- puppeteer-extra-plugin-stealth@2.6.5

Chromium | 88.0.4298.0 (Developer Build) (64-bit)

On latest version as I just installed it today since my own anti-botdetection failed. From what I see it's a new detection since today or yesterday.

Niek commented 3 years ago

Can you open page.goto('chrome://version') in headful mode so we can see the startup flags? I can't reproduce this.

berstend commented 3 years ago

Also if you could verify that the navigator.webdriver evasion has been loaded correctly by running your script like so:

DEBUG=puppeteer-extra,puppeteer-extra-plugin:* node myscript.js
riprekt commented 3 years ago
Chromium 88.0.4298.0 (Developer Build) (64-bit)
Revision e62cb7e3fc7c40548cef66cdf19d270535d9350b-refs/heads/master@{#818858}
OS Windows 10 OS Version 1909 (Build 18363.1198)
JavaScript V8 8.8.134
User Agent Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4298.0 Safari/537.36
Command Line "C:\Users\mathyro\node_modules\puppeteer.local-chromium\win64-818858\chrome-win\chrome.exe" --disable-background-networking --enable-features=NetworkService,NetworkServiceInProcess --disable-background-timer-throttling --disable-backgrounding-occluded-windows --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-background-pages --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=TranslateUI --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-backgrounding --disable-sync --force-color-profile=srgb --metrics-recording-only --no-first-run --enable-automation --password-store=basic --use-mock-keychain --enable-blink-features=IdleDetection --remote-debugging-port=0 --user-data-dir="C:\Users\mathyro\AppData\Local\Temp\puppeteer_dev_chrome_profile-T532Oe" --flag-switches-begin --flag-switches-end --file-url-path-alias="/gen=C:\Users\mathyro\node_modules\puppeteer.local-chromium\win64-818858\chrome-win\gen" about:blank
Executable Path C:\Users\mathyro\node_modules\puppeteer.local-chromium\win64-818858\chrome-win\chrome.exe
riprekt commented 3 years ago

@berstend I don't seem to manage to run the debug. I tried googling it, but no results. Is it a linux thing because I'm on windows?

berstend commented 3 years ago

Is it a linux thing because I'm on windows?

Ah yeah, the DEBUG example is an environment variable to control debug - windows does these things differently 😄

image

That fact that you're running Windows could be connected to the issue though.

riprekt commented 3 years ago

damn windows :D

puppeteer-extra no dependencies are missing +0ms puppeteer-extra orderPlugins:before [] +3ms puppeteer-extra orderPlugins:after [] +2ms puppeteer-extra-plugin:base:stealth Initialized. +0ms puppeteer-extra plugin registered stealth +1s

this output :-)

berstend commented 3 years ago

Windows (especially when not using stuff like WSL) isn't really supported (in a sense that it's not being tested) - there might be bugs here

riprekt commented 3 years ago

btw, I want to thank you guys, I really appreciate the fast responses!

berstend commented 3 years ago

this output :-)

That's all? 🤔 Seems like the stealth evasion plugins are not loading correctly..

berstend commented 3 years ago

It should look more like this:

image

riprekt commented 3 years ago

OMG! You're right... I accidentally placed puppeteer.use(StealthPlugin()) below puppeteer.launch()... I'm so stupid, sorry for wasting your time! D:

berstend commented 3 years ago

Haha, no worries :-) At least there's no general windows bug that went unnoticed for too long. 😄