macOS + Python (

[imthisguymike]

Installed selenium, selenium-stealth, beautifulsoup4 and also ran chromedriver

Traceback (most recent call last): File "scrape.py", line 7, in import requests ModuleNotFoundError: No module named 'requests'

[ryangavin]

The most recent version of the repo contains a requirements.txt now.

If you clone the repo you can run pip install -r requirements.txt to install all the dependencies which should fix this issue.

[imthisguymike]

@ryangavin It's further now, and I did run the pip3 install -r requirements.txt and it installed more but it seems to be wanting chromedriver.exe even though an exe like that wouldn't be on a mac

scrape.py:22: DeprecationWarning: executable_path has been deprecated, please pass in a Service object
  driver = webdriver.Chrome(options=options, executable_path=r"chromedriver.exe")
Traceback (most recent call last):
  File "/Users/bond/Library/Python/3.8/lib/python/site-packages/selenium/webdriver/common/service.py", line 71, in start
    self.process = subprocess.Popen(cmd, env=self.env,
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/subprocess.py", line 858, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/subprocess.py", line 1704, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'chromedriver.exe'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "scrape.py", line 22, in <module>
    driver = webdriver.Chrome(options=options, executable_path=r"chromedriver.exe")
  File "/Users/bond/Library/Python/3.8/lib/python/site-packages/selenium/webdriver/chrome/webdriver.py", line 70, in __init__
    super(WebDriver, self).__init__(DesiredCapabilities.CHROME['browserName'], "goog",
  File "/Users/bond/Library/Python/3.8/lib/python/site-packages/selenium/webdriver/chromium/webdriver.py", line 89, in __init__
    self.service.start()
  File "/Users/bond/Library/Python/3.8/lib/python/site-packages/selenium/webdriver/common/service.py", line 81, in start
    raise WebDriverException(
selenium.common.exceptions.WebDriverException: Message: 'chromedriver.exe' executable needs to be in PATH. Please see https://chromedriver.chromium.org/home

[AnsonLai]

Sadly I don't have a mac to test this out on. But I have hardcoded chromedriver.exe without thinking about the mac side of things. You can probably remove that line and install Chromedriver as follows: https://jonathansoma.com/lede/foundations-2017/classes/more-scraping/selenium/

I can't test this though. :(

[imthisguymike]

I was able to hack it to start the chromedriver but it doesn't seem to work. I might just create a windows VM to do this, thanks!

bond@Bonds-MacBook-Pro TeslaServiceManualScraper % python3 scrape.py
scrape.py:24: DeprecationWarning: executable_path has been deprecated, please pass in a Service object
  driver = webdriver.Chrome(options=options, executable_path=r'/usr/local/bin')
Traceback (most recent call last):
  File "scrape.py", line 24, in <module>
    driver = webdriver.Chrome(options=options, executable_path=r'/usr/local/bin')
  File "/Users/bond/Library/Python/3.8/lib/python/site-packages/selenium/webdriver/chrome/webdriver.py", line 70, in __init__
    super(WebDriver, self).__init__(DesiredCapabilities.CHROME['browserName'], "goog",
  File "/Users/bond/Library/Python/3.8/lib/python/site-packages/selenium/webdriver/chromium/webdriver.py", line 92, in __init__
    RemoteWebDriver.__init__(
  File "/Users/bond/Library/Python/3.8/lib/python/site-packages/selenium/webdriver/remote/webdriver.py", line 275, in __init__
    self.start_session(capabilities, browser_profile)
  File "/Users/bond/Library/Python/3.8/lib/python/site-packages/selenium/webdriver/remote/webdriver.py", line 365, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/Users/bond/Library/Python/3.8/lib/python/site-packages/selenium/webdriver/remote/webdriver.py", line 430, in execute
    self.error_handler.check_response(response)
  File "/Users/bond/Library/Python/3.8/lib/python/site-packages/selenium/webdriver/remote/errorhandler.py", line 247, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: session not created exception: Missing or invalid capabilities
  (Driver info: chromedriver=2.32.498537 (cb2f855cbc7b82e20387eaf9a43f6b99b6105061),platform=Mac OS X 10.16.0 x86_64)