Undetected_Chromedriver is not working

[de-coding-life]

Giving following error -

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\imdec\AppData\Local\Programs\Python\Python310\lib\multiprocessing\spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "C:\Users\imdec\AppData\Local\Programs\Python\Python310\lib\multiprocessing\spawn.py", line 125, in _main
    prepare(preparation_data)
  File "C:\Users\imdec\AppData\Local\Programs\Python\Python310\lib\multiprocessing\spawn.py", line 236, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\Users\imdec\AppData\Local\Programs\Python\Python310\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
    main_content = runpy.run_path(main_path,
  File "C:\Users\imdec\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 269, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "C:\Users\imdec\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 96, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "C:\Users\imdec\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "c:\Users\imdec\Desktop\script.py", line 12, in <module>
    driver = uc.Chrome()
  File "C:\Users\imdec\AppData\Local\Programs\Python\Python310\lib\site-packages\undetected_chromedriver\__init__.py", line 356, in __init__
    self.browser_pid = start_detached(
  File "C:\Users\imdec\AppData\Local\Programs\Python\Python310\lib\site-packages\undetected_chromedriver\dprocess.py", line 35, in start_detached
    ).start()
  File "C:\Users\imdec\AppData\Local\Programs\Python\Python310\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "C:\Users\imdec\AppData\Local\Programs\Python\Python310\lib\multiprocessing\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\imdec\AppData\Local\Programs\Python\Python310\lib\multiprocessing\context.py", line 327, in _Popen
    return Popen(process_obj)
  File "C:\Users\imdec\AppData\Local\Programs\Python\Python310\lib\multiprocessing\popen_spawn_win32.py", line 45, in __init__
    prep_data = spawn.get_preparation_data(process_obj._name)
  File "C:\Users\imdec\AppData\Local\Programs\Python\Python310\lib\multiprocessing\spawn.py", line 154, in get_preparation_data
    _check_not_importing_main()
  File "C:\Users\imdec\AppData\Local\Programs\Python\Python310\lib\multiprocessing\spawn.py", line 134, in _check_not_importing_main
    raise RuntimeError('''
RuntimeError:
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.

[ToasterUwU]

@de-coding-life I found a fix for that!

Had the same issue and googled what the cause is. Link to the explanation: Link

All you need to do is to put all your code into this:

if __name__ == "__main__":
    # Code here

[BrightChanges]

@ToasterUwU But what should I do when I want call the #Code here from another file...? Let say I have a Python class with a function that calls #Code something that looks like something below:

import undetected_chromedriver as uc

class Bot(uc.Chrome):
    def __init__(self, driver = uc.Chrome()):
        self.driver = driver
        super(Bot,self).__init__()

    def get_images(self):
        self.driver.get(URL)

And I want to call the method get_images() from the class above from another file like below:

from file.file import Bot

with Bot() as bot:
   bot.get_images()

However, I'm meeting the same freeze_support() error. I understand that you can put if name == "main": before the code, but in my case I can't do that... Do you know how to use undetected_chromedriver without causing the error in my case? I looks around a lot but still could not find the answer. I would greatly appreciate it if you could help me. Thank you.

[ToasterUwU]

@ToasterUwU But what should I do when I want call the #Code here from another file...? Let say I have a Python class with a function that calls #Code something that looks like something below:

import undetected_chromedriver as uc

class Bot(uc.Chrome):
    def __init__(self, driver = uc.Chrome()):
        self.driver = driver
        super(Bot,self).__init__()

    def get_images(self):
        self.driver.get(URL)

And I want to call the method get_images() from the class above from another file like below:

from file.file import Bot

with Bot() as bot:
   bot.get_images()

However, I'm meeting the same freeze_support() error. I understand that you can put if name == "main": before the code, but in my case I can't do that... Do you know how to use undetected_chromedriver without causing the error in my case? I looks around a lot but still could not find the answer. I would greatly appreciate it if you could help me. Thank you.

@BrightChanges i don't really see your problem. All you have to do is put everything in that if statement. Meaning that if you put the imports into that if statement as well, it will have that effect on all things you import as well.

So try putting every single thing you do into that if statement, including all the imports.

If that doesn't work, ping me again and I will try to help you more.

[BrightChanges]

@ToasterUwU Thank you for helping me. Right, but that means I need to include the if statement and everything in one file (bot.py), which doesn't allow me to run everything of my program from a separate file... Isn't there any way so I can keep this file (run.py) that run some code like below while allowing undetected chromedriver to work?

from file.file import Bot

with Bot() as bot:
   bot.get_images()

[ToasterUwU]

@ToasterUwU Thank you for helping me. Right, but that means I need to include the if statement and everything in one file (bot.py), which doesn't allow me to run everything of my program from a separate file... Isn't there any way so I can keep this file (run.py) that run some code like below while allowing undetected chromedriver to work?

from file.file import Bot

with Bot() as bot:
   bot.get_images()

@BrightChanges i think I'm missing context.. Can you show all the code you have? Or at least it's ground structure, and the error message as well?

Because right now I don't really understand what the problem is.

If you like to we can also make a short discord call about this. Add me if you want to do this, rather then texting back and forth.

Aki ToasterUwU#0001

[QIN2DIM]

@BrightChanges This is a very basic question.You can follow the code.

# -*- coding: utf-8 -*-
# Time       : 2022/3/1 23:05
# Author     : QIN2DIM
# Github     : https://github.com/QIN2DIM
# Description: sop.py
from alpha import get_challenge_ctx

def demo():
    ctx = get_challenge_ctx(silence=True)
    try:
        ctx.get("https://www.google.com")
        print(ctx.title)
    finally:
        ctx.quit()

if __name__ == '__main__':
    demo()

# -*- coding: utf-8 -*-
# Time       : 2022/3/1 23:05
# Author     : QIN2DIM
# Github     : https://github.com/QIN2DIM
# Description: alpha.py
from typing import Optional

import undetected_chromedriver as uc

def get_challenge_ctx(silence: Optional[bool] = None):
    return uc.Chrome(headless=silence)

[BrightChanges]

@ToasterUwU @QIN2DIM Thank you for your help but I was able to solve it!

Below is my solution:

1.bot.py:

import undetected_chromedriver as uc

class Bot(uc.Chrome):
    def __init__(self, teardown=False):
        self.teardown = teardown
        super(Bot,self).__init__()

    def __exit__(self, exc_type, exc_val, exc_tb):
        if self.teardown:
            self.quit()

    def __enter__(self):
        return self

    def get_images(self, driver):
        driver.get(URL)

2.run.py:

from file.file import Bot
import undetected_chromedriver as uc

if __name__ == "__main__":
      driver = uc.Chrome()
      with Bot() as bot:
          bot.get_images()

Again, thank you!

[ToasterUwU]

@ToasterUwU @QIN2DIM Thank you for your help but I was able to solve it!

Below is my solution:

1.bot.py:

import undetected_chromedriver as uc

class Bot(uc.Chrome):
    def __init__(self, teardown=False):
        self.teardown = teardown
        super(Bot,self).__init__()

    def __exit__(self, exc_type, exc_val, exc_tb):
        if self.teardown:
            self.quit()

    def __enter__(self):
        return self

    def get_images(self, driver):
        driver.get(URL)

2.run.py:

from file.file import Bot
import undetected_chromedriver as uc

if __name__ == "__main__":
      driver = uc.Chrome()
      with Bot() as bot:
          bot.get_images()

Again, thank you!

Jup, that's exactly what I was trying to explain.

Happy you got it working now.

[ToasterUwU]

@ultrafunkamsterdam ready to close this issue

[Itrasadalish]

@ToasterUwU Hey currently having a similar issue I think.

For me, I am having problems using the Undetected_Chromedriver in combination with flask. I guess the issue has something to do with the multiprocessing that is taking place in Undetected_Chromedriver.

@app.route('/')
  def homepage():

    # Here I would call my class and function that uses Undetected_Chromedriver
    # This class returns a list with items that I want to add to an SQLite database.
    return `render_template("index.html")

  if __name == __"main"__
      app.run()

  class :
      def __init__(self, ):

    def (self):
        if __name__ == '__main__':

            driver = uc.Chrome()      
            driver.get(self.link)

            return list

So I am calling if name == 'main': twice. In my case, the Flask server still works, but the imported Undetected_Chromedriver that resides in another python file, get executed right at the beginning and gives back errors.

I am not really sure where to look for solutions. Maybe you have an idea.

[ToasterUwU]

@ToasterUwU Hey currently having a similar issue I think.

For me, I am having problems using the Undetected_Chromedriver in combination with flask. I guess the issue has something to do with the multiprocessing that is taking place in Undetected_Chromedriver.

@app.route('/')
  def homepage():

    # Here I would call my class and function that uses Undetected_Chromedriver
    # This class returns a list with items that I want to add to an SQLite database.
    return `render_template("index.html")

  if __name == __"main"__
      app.run()

  class :
      def __init__(self, ):

    def (self):
        if __name__ == '__main__':

            driver = uc.Chrome()      
            driver.get(self.link)

            return list

So I am calling if name == 'main': twice. In my case, the Flask server still works, but the imported Undetected_Chromedriver that resides in another python file, get executed right at the beginning and gives back errors.

I am not really sure where to look for solutions. Maybe you have an idea.

@Itrasadalish The code snippets you send are not really helpful. Meaning they are formatted in a way that can't work and also a function is straight up missing a name.

I would try to put all the code that does anything into that name == "main" thing.

So everything except class and function defenitions and imports come into that thing. Making the Chrome window, running the flask server, everything.

[Itrasadalish]

@ToasterUwU Here is a more detailed code example:

app = Flask(__name__)

app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///add.db'
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False
db = SQLAlchemy(app)

class Add(db.Model):
    __tablename__ = "add"
    id = db.Column(db.Integer, primary_key=True)
    date = db.Column(db.String(100), nullable=False)
    name= db.Column(db.String(100), unique=True, nullable=False)

@app.route('/')
def homepage():
    return render_template("index.html")

@app.route('/add', methods=["GET", "POST"])
def add():
    form = Form()
    if form.validate_on_submit():
        if regex.search(r'\p{Han}', form.add.data):

            if Add.query.filter(Add.add== form.add.data).first():
                flash("Already exist!")
                return redirect(url_for("add"))

            else:
                user_info = AddInfo(form.add.data)        # <---- This is where the important function is called.
                general_info = user_info.general_info()

                new_add = Add(
                    date=date.today().strftime("%B %d, %Y"),
                    title=general_info[0],
                )
                db.session.add(new_add)
                db.session.commit()

                return redirect(url_for("homepage"))

        else:
            flash("error")

    return render_template("add.html", form=form)

if __name__ == "__main__":
    app.run(debug=True)

Above would be the snippet for my Flask application. The part that makes problems is where it calls the class AddInfo. AddInfo looks like this:

class AddInfo:
    def __init__(self, add):
        self.add= add
        self.link = # link #
        self.source_page = None

    def general_info(self):
        if __name__ == "__main__":    # With this here this function will return an emty list later in the Flask-Apllication.    
              driver = uc.Chrome()     # Without it I will instantly get an error when starting flask
              driver.get(self.link)

              time.sleep(2)
              choose = driver.find_element_by_class_name()
              choose.click()
              time.sleep(2)

              new_window = driver.window_handles[1]
              driver.switch_to.window(new_window)
              self.source_page = driver.page_source

              html_general = BeautifulSoup(self.source_page, 'html.parser')

              add= html_general.find('h1').text  # etc.

              general_info_list = [add,..,etc.]

              return general_info_list

I deleted some parts of the original code, but this should make it easy to replicate. At least I hope so.

The problem here is, that I can't really call the class function general info in my flask application, because I have to give it "if name == "main"" . If I do give it that, it just runs by starting the main.py, but when it doesn't get the input from the user from the submitted form the function will not work.

I don't really understand so much about how multiprocessing works, so it's quite hard to debug.

[ToasterUwU]

@Itrasadalish Try putting everything under if __name__ == "__main__": like this:

if __name__ == "__main__":
    class AddInfo:
        def __init__(self, add):
            self.add= add
            self.link = # link
            self.source_page = None

        def general_info(self):
            if __name__ == "__main__":    # With this here this function will return an emty list later in the Flask-Apllication.    
                driver = uc.Chrome()     # Without it I will instantly get an error when starting flask
                driver.get(self.link)

                time.sleep(2)
                choose = driver.find_element_by_class_name()
                choose.click()
                time.sleep(2)

                new_window = driver.window_handles[1]
                driver.switch_to.window(new_window)
                self.source_page = driver.page_source

                html_general = BeautifulSoup(self.source_page, 'html.parser')

                add= html_general.find('h1').text  # etc.

                general_info_list = [add,..,etc.]

                return general_info_list

    app = Flask(__name__)

    app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///add.db'
    app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False
    db = SQLAlchemy(app)

    class Add(db.Model):
        __tablename__ = "add"
        id = db.Column(db.Integer, primary_key=True)
        date = db.Column(db.String(100), nullable=False)
        name= db.Column(db.String(100), unique=True, nullable=False)

    @app.route('/')
    def homepage():
        return render_template("index.html")

    @app.route('/add', methods=["GET", "POST"])
    def add():
        form = Form()
        if form.validate_on_submit():
            if regex.search(r'\p{Han}', form.add.data):

                if Add.query.filter(Add.add== form.add.data).first():
                    flash("Already exist!")
                    return redirect(url_for("add"))

                else:
                    user_info = AddInfo(form.add.data)        # <---- This is where the important function is called.
                    general_info = user_info.general_info()

                    new_add = Add(
                        date=date.today().strftime("%B %d, %Y"),
                        title=general_info[0],
                    )
                    db.session.add(new_add)
                    db.session.commit()

                    return redirect(url_for("homepage"))

            else:
                flash("error")

        return render_template("add.html", form=form)

    app.run(debug=True)