Work with `pyautogui` and `seleniumbase` in a docker container

[chahak13]

Hello,

Thank you for your work on this project. I'm trying to run seleniumbase and pyautogui together in a linux docker container and am running into issues. On using base selenium, it works fine but stops working when I switch it out for seleniumbase. Here are my two scripts.

With selenium: This opens a new tab and goes to duckduckgo.

import pyautogui
import os
import platform
import Xlib.display
from time import sleep

from selenium import webdriver
from seleniumbase import SB, Driver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options as ChromeOptions
from sbvirtualdisplay import Display

ctrl_cmd = "ctrl" if platform.system() == "Linux" else "command"

display = None
if platform.system() == "Linux":
    display = Display(visible=False, size=(1920, 1080))
    display.start()

    pyautogui._pyautogui_x11._display = Xlib.display.Display(os.environ["DISPLAY"])

options = ChromeOptions()
options.add_argument("--no-sandbox")
browser = webdriver.Chrome(options=options)
# browser = Driver()
browser.get("https://google.com/")
pyautogui.screenshot("at_google.png")
pyautogui.hotkey(ctrl_cmd, "t")
pyautogui.hotkey(ctrl_cmd, "l")
pyautogui.typewrite("https://duckduckgo.com/")
pyautogui.hotkey("return")
sleep(2)
pyautogui.screenshot("at_ddg.png")
print(browser.title)
browser.quit()

if display:
    display.stop()

And with seleniumbase: This does not do anything.

import pyautogui
import os
import platform
import Xlib.display
from time import sleep

from selenium import webdriver
from seleniumbase import SB, Driver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options as ChromeOptions
from sbvirtualdisplay import Display

ctrl_cmd = "ctrl" if platform.system() == "Linux" else "command"

display = None
if platform.system() == "Linux":
    display = Display(visible=False, size=(1920, 1080))
    display.start()

    pyautogui._pyautogui_x11._display = Xlib.display.Display(os.environ["DISPLAY"])

with SB(xvfb=True) as sb:
    sb.driver.get("https://google.com/")
    pyautogui.screenshot("at_google.png")
    pyautogui.hotkey(ctrl_cmd, "t")
    pyautogui.hotkey(ctrl_cmd, "l")
    pyautogui.typewrite("https://duckduckgo.com/")
    pyautogui.hotkey("return")
    sleep(2)
    pyautogui.screenshot("at_ddg.png")
    print(sb.driver.title)
    sb.driver.quit()

if display:
    display.stop()

Could you please help me figure out what could be going on here. I'm happy to provide more details about anything else needed.

NOTE: Both of these versions work fine locally on a mac (which would not use the virtual display afaik) so maybe it could be related to that?

[mdmintz]

Where does it stop working, and what about a stack trace? Is it possible your issue is coming from pyautogui since that's what you used? It might also be that the Xlib.display you used is slightly different from how Xvfb works. Xfvb allows you to run a headed browser in a headless environment, but that environment is still headless, so you might not be able to fully interact with it using pyautogui.

Note that the driver is quit automatically at the end of the with block, which means you don't need to call sb.driver.quit() manually there.

Also, it looks like what you're trying to do can be achieved with pure SeleniumBase code (no pyautogui needed) so you may want to consider doing that instead:

from seleniumbase import SB

with SB() as sb:
    sb.open("https://google.com/")
    sb.save_screenshot("at_google.png")
    sb.type('[title="Search"]', "https://duckduckgo.com/\n")
    sb.save_screenshot("at_ddg.png")
    print(sb.driver.title)

[chahak13]

Thanks for the quick response! I was mainly concerned because in the same environment pyautogui works if it's working on the selenium browser but not here.

Where does it stop working, and what about a stack trace?

It does not error out but the pyautogui steps just don't register? I will also create an issue on there but since it stops working on changing SB, I was confused.

Also, regarding your notes, I'm aware of the quit function, just threw something together quickly and hence missed that. Also, I understand seleniumbase can do all of it on it's own. This one's just a dummy script that's sort of representative of my original script which does indeed require pyautogui :)

I've not used xvfb much so am not very well versed. Do you have any suggestions regarding what I can do on this?

[mdmintz]

You could take this Driver() manager script and swap out the sbvirtualdisplay part with your Xlib.display code:

from sbvirtualdisplay import Display
from seleniumbase import Driver

display = Display(visible=0, size=(1440, 1880))
display.start()

driver = Driver(headed=True)
driver.get("https://google.com")
# ...
driver.quit()

display.stop()

But if the reason you need pyautogui is to bypass CAPTCHAs, you can just do that with SeleniumBase UC Mode.

[chahak13]

Thank you! That helped me fix the script that I had. (Was actually just missing headed=True for some reason 🤦🏼‍♂️ )

Regarding UC Mode, I'm already using it, thank you for including that! pyautogui is doing some other stuff.

[mdmintz]

Great, I'll close this ticket. Don't forget to show support by starring the repo on GitHub: https://github.com/seleniumbase/SeleniumBase/stargazers

[chahak13]

Hi @mdmintz,

Sorry, don't mean to reopen this but is there a way to do the same with the SB context? It works fine with the Driver but not when working inside with SB. Thanks!

[mdmintz]

Are you using the same options? The SB() format automatically activates Xvfb, whereas the Driver() format expects you'll add your own code for that, such as with sbvirtualdisplay, etc.

[chahak13]

I'm using it with uc=True and headed=True, yes. Would adding xvfb=False explicitly be the way?

[mdmintz]

I would try out different combinations of settings and see if that solves the issue for you. Otherwise, use the Driver() format, which seems to work for you.

[chahak13]

I got it to work with xvfb=False explicitly, thanks!