MatthewChatham / glassdoor-review-scraper

Scrape reviews from Glassdoor
BSD 2-Clause "Simplified" License
177 stars 252 forks source link

Chrome web driver arg chrome_options is deprecated #9

Open yihaozhadan opened 5 years ago

yihaozhadan commented 5 years ago

Chrome web driver arg chrome_options is deprecated. Please use "option" instead. For example, in main.py line 382, change to

browser = wd.Chrome(options=chrome_options)

Here is the reference webdriver.py.

By the way, may I be a contributor to your repository?

mchatham-fk commented 5 years ago

Hey @yihaozhadan, thanks for the suggestion! If you fork the repo, you can make a PR from your fork. Thanks!

liudi1025 commented 5 years ago

Hi, after changing 'chrome_options' into 'options', I still encounter the problem with error message: "The application was unable to start correctly(0xc000007b)." I followed all the steps in the README.md. However, still not working here.

Below is the error message:

2019-04-21 12:48:55,882 INFO 377 :main.py(17992) - Configuring browser Traceback (most recent call last): File "main.py", line 411, in browser = get_browser() File "main.py", line 382, in get_browser browser = wd.Chrome(options=chrome_options) File "C:\Users\liudi\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 81, in init desired_capabilities=desired_capabilities) File "C:\Users\liudi\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 157, in init self.start_session(capabilities, browser_profile) File "C:\Users\liudi\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 252, in start_session response = self.execute(Command.NEW_SESSION, parameters) File "C:\Users\liudi\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute self.error_handler.check_response(response) File "C:\Users\liudi\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.WebDriverException: Message: unknown error: DevToolsActivePort file doesn't exist (Driver info: chromedriver=73.0.3683.68 (47787ec04b6e38e22703e856e101e840b65afe72),platform=Windows NT 10.0.17134 x86_64)

yihaozhadan commented 5 years ago

@liudi1025 Did you download ChromeDirver? When you check out the repository. It requires to put chromedriver.exe and your Glassdoor credential file under the root folder. The structure looks like:

- chromedriver.exe
- main.py
- README.md
- requirements.txt
- schema.py
- secret.json
liudi1025 commented 5 years ago

Yes, I did include the ChromeDriver.

liudi1025 commented 5 years ago

Now I am getting the error message as:

2019-04-21 13:30:18,740 INFO 377 :main.py(19096) - Configuring browser Traceback (most recent call last): File "main.py", line 411, in browser = get_browser() File "main.py", line 382, in get_browser browser = wd.Chrome(options=chrome_options) File "C:\Users\liudi\Anaconda3\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 81, in init desired_capabilities=desired_capabilities) File "C:\Users\liudi\Anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 157, in init self.start_session(capabilities, browser_profile) File "C:\Users\liudi\Anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 252, in start_session response = self.execute(Command.NEW_SESSION, parameters) File "C:\Users\liudi\Anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute self.error_handler.check_response(response) File "C:\Users\liudi\Anaconda3\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: crashed (unknown error: DevToolsActivePort file doesn't exist) (The process started from chrome location C:\Users\liudi\AppData\Local\Google\Chrome\Application\chrome.exe is no longer running, so ChromeDriver is assuming that Chrome has crashed.) (Driver info: chromedriver=73.0.3683.68 (47787ec04b6e38e22703e856e101e840b65afe72),platform=Windows NT 10.0.17134 x86_64)

liudi1025 commented 5 years ago

I got what happened here: there are two chrome.exe in my computer and the path direct me to the crashed version of chrome... Thanks!@yihaozhadan

yihaozhadan commented 5 years ago

@liudi1025 It's good to hear that. No problem.

MatthewChatham commented 5 years ago

@yihaozhadan can you please make a PR to update this? Thanks!

yihaozhadan commented 5 years ago

@MatthewChatham Created pull request.

wenxin-lab commented 4 years ago

Hi, I tried the code, and got this error:

2019-09-29 15:41:04,215 INFO 377 :main.py(10918) - Configuring browser Traceback (most recent call last): File "main.py", line 412, in browser = get_browser() File "main.py", line 382, in get_browser browser = wd.Chrome(options=chrome_options) File "/usr/local/var/pyenv/versions/anaconda3-2019.03/lib/python3.7/site-packages/selenium/webdriver/chrome/webdriver.py", line 81, in init desired_capabilities=desired_capabilities) File "/usr/local/var/pyenv/versions/anaconda3-2019.03/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in init self.start_session(capabilities, browser_profile) File "/usr/local/var/pyenv/versions/anaconda3-2019.03/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session response = self.execute(Command.NEW_SESSION, parameters) File "/usr/local/var/pyenv/versions/anaconda3-2019.03/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute self.error_handler.check_response(response) File "/usr/local/var/pyenv/versions/anaconda3-2019.03/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.SessionNotCreatedException: Message: session not created: This version of ChromeDriver only supports Chrome version 78

But my chrome version is 78, so I have no idea how to fix it...