Selenium script giving exception

[akash-ai]

I've developed a selenium script for retrieving Google Trends data. However, occasionally, the graph fails to load correctly based on my inputs, preventing the proper download of data. Despite employing the 'fake_useragent' library and obtaining cookies from 'requests' library, Selenium struggles to load the graph correctly. Any insights or suggestions would be appreciated. Thanks.

Code:

`from selenium.webdriver.firefox.service import Service as FirefoxService from selenium.webdriver.firefox.options import Options as FirefoxOptions from fake_useragent import UserAgent

def get_selenium(output_dir): firefox_options = FirefoxOptions() ua = UserAgent() user_agent = ua.random service = FirefoxService(executable_path='geckodriver.exe') prefs = { 'download.default_directory': output_dir, 'download.prompt_for_download': False, 'download.directory_upgrade': True, 'browser.download.folderList': 2, 'browser.download.manager.showWhenStarting': False, 'browser.download.manager.showAlertOnComplete': False, 'browser.download.manager.useWindow': False, 'browser.helperApps.neverAsk.saveToDisk': 'application/csv,text/csv', } firefox_options.set_preference('browser.download.manager.useWindow', False) firefox_options.set_preference('browser.download.manager.focusWhenStarting', False) firefox_options.set_preference('browser.download.manager.showAlertOnComplete', False) firefox_options.set_preference('browser.download.manager.closeWhenDone', True) for key, value in prefs.items(): firefox_options.set_preference(key, value) firefox_options.add_argument('--headless') firefox_options.add_argument('--window-size=1980,1080') firefox_options.add_argument(f'--user-agent={user_agent}') driver = webdriver.Firefox(service=service, options=firefox_options) driver.implicitly_wait(5) return driver`

[williamegomezo]

Use seleniumbase with undetected driver, it works for me.

[davidgaut]

I also get a similar problem, whether or not I use seleniumbase with the undetected driver.