How to open selenium browser & make drive headful

[Kamal-Moha]

I know that by default the driver is --headless meaning it's not able to open browser in colab. I'm trying to add custom option so as to make driver headful and show the interactive selenium browser.

import google_colab_selenium as gs
from selenium.webdriver.chrome.options import Options

custom_options = Options()
# Add custom options
custom_options.headless = False

driver = gs.Chrome(options=custom_options)

I have also tried; custom_options.add_argument('--headless=False')

After running all of the above, I expected drive to be headful and display browser.

from time import sleep
driver.get('https://www.chelseafc.com/en')
sleep(3)
print(driver.title)
driver.quit()

But I didn't see any browser opening up. Please help

[jpjacobpadilla]

Hi, are you trying to use a virtual display such as xvbf inside of the notebook?

This package sets the Selenium instance to be headless, since I don't see much of a point in making it headful (but feel free to re-open this issue to correct me if I'm wrong). At the moment, I don't really see the need to implement a virtual display into this project.

[Kamal-Moha]

@jpjacobpadilla one big reason to use selenium over other forms of scraping is the ability of selenium to showcase/display the browser of what's happening. Essentially viewing the browser display enables you see what your code is doing, what buttons are you clicking, etc which are very important when working with selenium. When using Selenium Chromedriver in VS Code, its able to open that browser in a new tab, the only issue is that I don't mostly use VS Code as colab is my favorite environment. I came across this library thinking its able to do everything that we can do with selenium.

I think showing that display browser is very helpful

[jpjacobpadilla]

Unfortunately, I don't think it's possible to stream a live view of the Selenium browser into the output of a notebook cell :/.

One solution that I came up with is to take screenshots of the browser every second and then display those in the cell output (clearing the previous one each time, so that there's only ever one image in the output). What do you think of that idea?

At the end of the day, I would advise developing a Selenium program on your computer and then moving it to a Colab Notebook. This is because even if it were possible to display a live view of what the browser was doing in Google Colab, the main piece that you're missing is the ability to use Inspect Element/ Developer Tools in the Chrome instance, which I don't see ever being possible in a Colab notebook.