search

clemfromspace / scrapy-selenium

Scrapy middleware to handle javascript pages using selenium
Do What The F*ck You Want To Public License
919 stars 348 forks source link

Chrome headless mode isn't working #115

Open shengyilu opened 2 years ago

shengyilu commented 2 years ago

I followed the instruction to edit settings.py as :

SELENIUM_DRIVER_NAME = 'chrome' SELENIUM_DRIVER_EXECUTABLE_PATH = '/Users/****/chromedriver' SELENIUM_DRIVER_ARGUMENTS=['--headless']

However, the Chrome will be launched every time I run the spider. BTW, with only Selenium, headless mode was working correctly: options = Options() options.add_argument("--headless")
driver = webdriver.Chrome(service = Service(ChromeDriverManager().install()), options = options)

Does anyone face the same problem? Need some help.

deduble commented 2 years ago

from shutil import which ChromeDriverManager().install() also downloads and returns path. so this was working for me

SELENIUM_DRIVER_NAME = 'chrome' SELENIUM_DRIVER_EXECUTABLE_PATH = which(ChromeDriverManager().install()) SELENIUM_DRIVER_ARGUMENTS=['--headless']