search

bellingcat / auto-archiver

Automatically archive links to videos, images, and social media content from Google Sheets (and more).
https://pypi.org/project/auto-archiver/
MIT License
581 stars 61 forks source link

Page load timeout #24

Closed djhmateer closed 2 years ago

djhmateer commented 2 years ago

Nice work on fixes in the PR https://github.com/bellingcat/auto-archiver/pull/21/commits/398f296789ad4c38c39ab9473e925c72a40fe718

I'm using a larger page load timeout of 120 seconds for some tricky long running queries ones (specifically telegram) which seems to work.

# I think 120 is seconds - the docs are ambiguous
driver.set_page_load_timeout(120)

https://www.selenium.dev/selenium/docs/api/py/webdriver_remote/selenium.webdriver.remote.webdriver.html#selenium.webdriver.remote.webdriver.WebDriver.set_page_load_timeout

msramalho commented 2 years ago

yup, the docs are a bit incomplete but should be seconds. Perhaps we could extract this wait time into a config. Thanks for sharing the values which work for your use-case.

djhmateer commented 2 years ago

Miguel has done it in latest updates, so settable via json config.