gigacodr commented 2 years ago

Hi Ultrafunkamsterdam, you have created such a useful tool for thousands of developers around the world. Please consider continuing your project!

mcflem06 commented 1 year ago

Is he no longer supporting the project?

gigacodr commented 1 year ago

Is he no longer supporting the project?

The project is not working properly right now and he usually committed frequently.

Edit: Accidentally closed the issue

mcflem06 commented 1 year ago

Weird - I've been relying on it for a bit and it's working fine for me.

gigacodr commented 1 year ago

Weird - I've been relying on it for a bit and it's working fine for me.

Using undetected_chromedriver with chrome profiles is broken.

For instance, you are not able to control the driver anymore when connected with a profile, but without is fine.

MelodyEars commented 1 year ago

This is true. When you start the driver, as in the readme, the driver loads the ...\Default folder. If you add --profile-directory with more than 1 profiles, it will run the first profile instead of other profiles. what to do? Any ideas? thank you.

gigacodr commented 1 year ago

This is true. When you start the driver, as in the readme, the driver loads the ...\Default folder. If you add --profile-directory with more than 1 profiles, it will run the first profile instead of other profiles. what to do? Any ideas? thank you.

This block shows how to do it when it does work:

import undetected_chromedriver as uc

chrome_options = uc.ChromeOptions()

profile = "C:\\Users\\_USER_NAME_\\AppData\\Local\\Google\Chrome\\User Data\\Profile 1"
chrome_options.add_argument(f"--user-data-dir={profile}")

driver = uc.Chrome(options = chrome_options, use_subprocess=True, version_main = 105)

The problem is the version of Chrome I believe, try an older version, preferably <= 97.

Also, Chrome tries very hard to force an update, make sure to disable that. As of right now, I do not have a permanent solution for stopping the updates.

gigacodr commented 1 year ago

Weird - I've been relying on it for a bit and it's working fine for me.

What version of Chrome are you on?

mcflem06 commented 1 year ago

106, but not using profiles.

On Thu, Sep 29, 2022 at 5:00 PM wazoooo @.***> wrote:

Weird - I've been relying on it for a bit and it's working fine for me.

What version of Chrome are you on?

— Reply to this email directly, view it on GitHub https://github.com/ultrafunkamsterdam/undetected-chromedriver/issues/820#issuecomment-1262810363, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAUEJJHE7OFJTUQ5HBH67XLWAX7OVANCNFSM6AAAAAAQTSSIXI . You are receiving this because you commented.Message ID: @.*** com>

MelodyEars commented 1 year ago

This block shows how to do it when it does work:
import undetected_chromedriver as uc

chrome_options = uc.ChromeOptions()

profile = "C:\\Users\\_USER_NAME_\\AppData\\Local\\Google\Chrome\\User Data\\Profile 1"
chrome_options.add_argument(f"--user-data-dir={profile}")

driver = uc.Chrome(options = chrome_options, use_subprocess=True, version_main = 105)
The problem is the version of Chrome I believe, try an older version, preferably <= 97.

Also, Chrome tries very hard to force an update, make sure to disable that. As of right now, I do not have a permanent solution for stopping the updates.

Thank you very much for your response!

But your answer still doesn't work for me. Version Google: 96

path_to_profile = "C:\\Users\\_USER_NAME_\\AppData\\Local\\Google\Chrome\\User Data\\Profile 1" Not in this way: chrome_options.add_argument(f"--user-data-dir={path_to_profile}")

not in this: options.user_data_dir = path_to_profile

and not in this: driver = uc.Chrome(user_data_dir=path_to_profile)

Like I said the only thing that works for me:(This )

def driver(profile=str, browser_executable_path=None):
    """
    Call driver via undetected_driver;
    if you pass profile:
        profile: your (Profile num)
    else:
        driver opens the incognito webpages and deletes cookies.
        Then you can use authorization on the YouTube
    return: driver
    """
    options = uc.ChromeOptions()
    if profile:

        # match on windows 10
        options.add_argument(r"--user-data-dir=C:\Users\Username\AppData\Local\Google\Chrome\User Data")
        options.add_argument(f"profile-directory={profile}")

        # skip popups
        options.add_argument(
            '--no-first-run --no-service-autorun --password-store=basic --no-default-browser-check'
        )
        DRIVER = uc.Chrome(options,
                                browser_executable_path=browser_executable_path,
                                version_main=96,
                                use_subprocess=True)
    else:
        options.add_argument("--incognito")
        DRIVER = uc.Chrome(options,
                                version_main=96,
                                use_subprocess=True)
        DRIVER.delete_all_cookies()
    DRIVER.maximize_window()
    return DRIVER

driver("Profile 1", "path_to_chrome.exe")

A project "Uploading shorts to YouTube" that I will develop here, but there is no version with profile loading yet.

UPD. How to turn off google updates, I found the solution here.(It doesn't work)

UPD This code does not work, below is a different code from the owner of the repository. I left this code to show how not to do it.

gigacodr commented 1 year ago

@ArtDanger Thanks for your response!

I will try to do some testing regarding stopping the updates using your method. Will it work even if you restart your computer? Once I restarted it kept enabling those services and updated Chrome.

gigacodr commented 1 year ago

106, but not using profiles. … On Thu, Sep 29, 2022 at 5:00 PM wazoooo @.> wrote: Weird - I've been relying on it for a bit and it's working fine for me. What version of Chrome are you on? — Reply to this email directly, view it on GitHub <#820 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAUEJJHE7OFJTUQ5HBH67XLWAX7OVANCNFSM6AAAAAAQTSSIXI . You are receiving this because you commented.Message ID: @. com>

Then without profiles it should work on any version!

MelodyEars commented 1 year ago

I will try to do some testing regarding stopping the updates using your method. Will it work even if you restart your computer? Once I restarted it kept enabling those services and updated Chrome.

Sorry @wazoooo, I was wrong. This does not work after rebooting the OS. But it worked:

Turn off Microsoft Edge updates. because right after reboot I had google version like microsoft edge. I think they are related.

Download app Autoruns :

Thank you for your attention.

Do you have any solutions with loading multiple profiles?

ultrafunkamsterdam commented 1 year ago

Wazzzaaaaaaaaaaaaaap

ultrafunkamsterdam commented 1 year ago

You CAN only work with 1 profile a time, which may not be currently in use. ( also see background processes, and kill them before attempting to use the profile associated with it . This is a limitation of Chrome. Not UC.

ultrafunkamsterdam commented 1 year ago

I'm still working on the project, no worries. but i 'm overloaded with issues which aren't actual issues at all but wrong configs and custom plugins which arent even actually supported . Or being blocked by XYZ , and when seeing the code, or no accompanied code at all, i fullly understand they are getting blocked. So i don't even respond to those . Sorry i must have missed on this one. Things i'm working on atm is a full rewrite of the the tools , which can disable chromedriver binary at all. Which means implementing every possible method by hand.

MelodyEars commented 1 year ago

Guys, I'm stupid, everything is fine. I forgot to write -- before the chrome option.

options.add_argument(f"profile-directory={profile}")

I fixed the code now it works with multiple profiles in version 96, from the latest version of chrome, I did not check it.

def driver(profile=str, browser_executable_path=None):

    options = uc.ChromeOptions()

    # match on windows 10
    options.add_argument(r"--user-data-dir={os.environ['USERPROFILE']}\AppData\Local\Google\Chrome\User Data")
    options.add_argument(f"--profile-directory={profile}")

    DRIVER = uc.Chrome(options,
                            browser_executable_path=browser_executable_path,
                            version_main=96,
                            use_subprocess=True)

    DRIVER.maximize_window()
    return DRIVER

driver("Profile 1", "path_to_chrome.exe")

Full code implementation here

UPD

# skip popups
options.add_argument('--no-first-run --no-service-autorun --password-store=basic --no-default-browser-check')

removed these lines, read below why.

ultrafunkamsterdam commented 1 year ago

still working

I just retested the whole profile thing and working brilliant Here's what i did

steps:

launched a regular chrome, using the command line to create a new profile like so: "c:\Program Files\Google\Chrome Beta\Application\chrome.exe" --user-data-dir="d:\UC\User Data"

(for windows haters: yes, i did this on a windows machine, i got multiple systems and by accident i was working on something where i needed windows)
i logged in to one of the 500 google accounts i got
visited gmail
visited https://myaccount.google.com/

at this point, when i want to use this profile in chromedriver , i go like this (below code)

import undetected_chromedriver as uc

# the below (colons)  is ugly. but when using an interactive interpreter it saves you your fingers

import logging; logging.getLogger().setLevel(10); logger = logging.getLogger('undetected_chromedriver'); logger.setLevel(10); logger.addHandler(logging.FileHandler(filename='chrome.log'))

driver = uc.Chrome(user_data_dir="D:/UC/User Data", debug=True)
driver.get('https://gmail.com')   # works fine , no login needed whatsoever
driver.get('https://myaccount.google.com')  # works fine, no login required logged in
driver.exit()

ultrafunkamsterdam commented 1 year ago

Guys, I'm stupid, everything is fine. I forgot to write -- before the chrome option. options.add_argument(f"profile->directory={profile}")

    options = uc.ChromeOptions()
    options.add_argument(r"--user-data-dir={os.environ['USERPROFILE']}\AppData\Local\Google\Chrome\User Data")
    options.add_argument(f"--profile-directory={profile}")
    # skip popups
    options.add_argument('--no-first-run --no-service-autorun --password-store=basic --no-default-browser-check')
    DRIVER = uc.Chrome(options,
                            browser_executable_path=browser_executable_path,
                            version_main=96,
                            use_subprocess=True)
    DRIVER.maximize_window()
    return DRIVER
driver("Profile 1", "path_to_chrome.exe")

MAN, why is everyone making it so hard for themselves. Just leave the damn uc.ChromeOptions() out, just like those:

options.add_argument('--no-first-run --no-service-autorun --password-store=basic --no-default-browser-check')

This COULD bring your driver in a state where is IS detectable. Every custom option you make might (and potentially WILL) be triggering detection.

I did not bake all required options in the code "just for fun"

there is a reason behind it. I wanted to be able to pass those silly WAF's. I did not want (and still don't) want to support all kinds of plugins, or functionality which could compromise the stealthness.

This project is called undetected_chromedriver and it has 2 missions:

1 - Preventing as much as possible the detections of million-dollar bot-evasion companies. It does a very darn-good job at this. When correctly used (more about that below)

2 - being able to do a simple import ( for instance in an interactive session) and just get going in 1 call.

CORRECLY USED ?

what does that MEAN? (might some of you think)

There are 100s of factors come into play which can cause detection. Including but not limited to

working on clean environments (clean? yes, clean as in you are not using the machine you also use for other sketchy stuff)
behave like a good citizen (or at least pretend to) (also when trying to scrape Nike)
do not flood servers.
do not enter your (mostly logon) pages directly (but first go see the homepage and navigate to the form from there. Some sites place cookies, or use CSRF tokens on their regular pages, which will not be present when you enter their login page directly (eg: /some/path/to/login and surely not on your fresh cookie-and-history-less browsers.
navigating at the speed of light. you'll be immediately be flagged as bot (just like the ip address (yes your residential or "home" one as well )
do not run this in a datacenter (vps, heroku, etc etc are also datacenters!)
datacenter ip ranges are quite known obviously and no self-respecting-human-it-guy is browsing in a freezing-cold datacenter seeking for nike sneakers at a rate of 50 visites a second ;)) those 5 employees working there probably have a non-datacenter line.
If you fucked up expect some captcha's the next couple of times, until you've shown "enough" human-like behavior to get unflagged. Your behaviours (also on other sites) will be remembered by those multi-million-dollar-companies who more or less "own" the internet..... (and you ).

recurring arguments regarding the above i hear a lot

but,but,but when i do regular chrome on same ip it works!

yep. that works. well done sherlock.

except these regular browser using different fingerprints AND contains a shitload of history and cookies which look sufficiently enough to assume a human is using it.

Just keep repeating the bad config/behavior for long enough and you will eventually also start noticing captcha's in your regular browser, visiting your favourite porn site , and that sucks.

Bottom line

Don't get me wrong. I know other people have other wishes, use cases and requirements, like plugins or custom configs, which i can fully understand (: hacking. story of my life :).

however, taking into account the objectives for this project, raising tickets won't be answered (at least not by me). Some got lucky thanks to the exteremely kind, and helpful @sebdelsol .

so you want x,y,z to work or using xyz proxies, plugins - bummer - your problem - fork the repo and make your own adjustments.

bottomest line:

I just don't have time for issues which aren't really issues ( yes, its a "real" issue for that person using it wrong ) but those are often people with characteristics like

never touched python before
not knowing google, or just to lazy to use the search functionality
regarding the above, searching in this repo's issues (closed and open) to look for valueale information would be a step too high.
or probably won't read readme's and docs at all.

So , that was about it for the rant. @wazoooo @ArtDanger this was not about you. You're sweet :heart:

gigacodr commented 1 year ago

Thank you @ultrafunkamsterdam !

ultrafunkamsterdam commented 1 year ago

One minor thing. You should load the chrome data root directory, and not the folders within, like Profile / default/ or whatsoever

gigacodr commented 1 year ago

One minor thing. You should load the chrome data root directory, and not the folders within, like Profile / default/ or whatsoever

Got it!

MelodyEars commented 1 year ago

One minor thing. You should load the chrome data root directory, and not the folders within, like Profile / default/ or whatsoever

Thank you! Based on your answer, I made a project that will help those who don't understand.

Eddie-254 commented 1 year ago

Can someone please explain how to log in twitch with undetected driver? Below is my code

`import undetected_chromedriver as uc from selenium import webdriver from selenium.webdriver.common.keys import Keys from selenium.webdriver.support import expected_conditions as EC from selenium.webdriver.common.by import By from selenium.webdriver.support.wait import WebDriverWait import time import os

import wget

os.environ['PATH'] += r"C:/Chromedriver"

options = uc.ChromeOptions() options.add_argument("user-data-dir=C:\Users\user\AppData\Local\Google\Chrome\User Data\Profile 2") driver = uc.Chrome(chrome_options=options) driver.set_window_size(1024, 600) driver.maximize_window()

open the webpage

driver.get("https://www.twitch.tv/brawlhalla")

target username

login = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#root > div > div.Layout-sc-nxg1ff-0.bSuLAT > nav > div > div.Layout-sc-nxg1ff-0.hWJFll > div.Layout-sc-nxg1ff-0.jwASrz > div > div.Layout-sc-nxg1ff-0.iLhppl.anon-user > div:nth-child(1) > button > div > div"))).click() username = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#login-username"))) password = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#password-input")))

enter username and password

username.clear() username.send_keys("3ddi6") password.clear() password.send_keys("vC%=xA!i&Z7bgUT")

target the login button and click it

button = WebDriverWait(driver, 2).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "body > div.ReactModalPortal > div > div > div > div > div > div.Layout-sc-nxg1ff-0.gmqBFP > div > div > div.Layout-sc-nxg1ff-0.gZaqky > form > div > div:nth-child(3) > button > div > div"))).click()

`

quentinjs commented 1 year ago

One minor thing. You should load the chrome data root directory, and not the folders within, like Profile / default/ or whatsoever

Thank you! Based on your answer, I made a project that will help those who don't understand.

Can you add some descriptive text to it, such as the purpose. I am sure those who know totally understand the purpose of the profiles, but if I don't then many others definitely don't either. Thanks.

arfathyahiya commented 10 months ago

Hi, when trying to fetch the driver cookies I'm getting this error It started appearing out of nowhere today

selenium.common.exceptions.WebDriverException: Message: unknown error: result.webdriverValue.value list is missing or empty in Runtime.callFunctionOn response
  (Session info: chrome=119.0.6045.105)

sdfgsdfgd commented 8 months ago

this thread is pinned at the issues page, can we remove it from the pinned threads at the top of all issues, or close this

@ultrafunkamsterdam leooooon

gigacodr commented 7 months ago

I will go ahead and close it. Thanks for the contributions.

ultrafunkamsterdam / undetected-chromedriver

We need you back Leon! #820

still working

I did not bake all required options in the code "just for fun"

CORRECLY USED ?

There are 100s of factors come into play which can cause detection. Including but not limited to

recurring arguments regarding the above i hear a lot

but,but,but when i do regular chrome on same ip it works!

Just keep repeating the bad config/behavior for long enough and you will eventually also start noticing captcha's in your regular browser, visiting your favourite porn site , and that sucks.

Bottom line

bottomest line:

import wget

open the webpage

target username

enter username and password

target the login button and click it