search

algolia / docsearch-scraper

DocSearch - Scraper
https://docsearch.algolia.com/
Other
309 stars 108 forks source link

Update steps and browser_handler.py #371

Closed tatiana-perry closed 6 years ago

tatiana-perry commented 6 years ago

I had some trouble getting this running. Since selenium does not work on the new versions of Firefox. I had to download a previous version. Then I started getting this error:

selenium.common.exceptions.WebDriverException: Message: Can't load the profile. Profile Dir: /var/folders/01/z68fg4h17y7fd8hjq331qmzw0000gp/T/tmp0UXJjA/webdriver-py-profilecopy If you specified a log_file in the FirefoxBinary constructor, check it for details.

In the scraper/source/config/browser_handler.py I needed to specify a location for the Firefox profile and create a new one.

In summary to get it running: Firefox: 41.0 Mac OS: 10.12.6 Create new user profile Add profile to browser_handler.py

profile = webdriver.FirefoxProfile('/Users/{user_name}/Library/Application 
  Support/Firefox/Profiles/{custom_profile_name}')
s-pace commented 6 years ago

:wave: @tatiana-perry

The recommended version is described here

If you install this precise one (v45.0.2), everything should work fine

tatiana-perry commented 6 years ago

Thanks @s-pace I did get it working and your fix works also.