MatthewChatham / glassdoor-review-scraper

Scrape reviews from Glassdoor
BSD 2-Clause "Simplified" License
172 stars 251 forks source link

NoSuchElementException - nable to locate element: {"method":"css selector","selector":"p"} #32

Closed beliakon closed 4 years ago

beliakon commented 4 years ago

I have this error and don't know how to fix it.

python main.py --headless --url "https://www.glassdoor.com/Overview/Working-at-Wells-Fargo-EI_IE8876.11,22.htm" --limit 5 -f wells_fargo_reviews.csv 2019-11-21 16:21:07,264 INFO 377 :main.py(19772) - Configuring browser

DevTools listening on ws://127.0.0.1:54285/devtools/browser/45fc8723-9a20-498a-864e-5b4544f795ab 2019-11-21 16:21:09,710 INFO 419 :main.py(19772) - Scraping up to 5 reviews. 2019-11-21 16:21:09,718 INFO 358 :main.py(19772) - Signing in to blah@blah.com 2019-11-21 16:21:15,642 INFO 339 :main.py(19772) - Navigating to company reviews 2019-11-21 16:21:37,112 INFO 286 :main.py(19772) - Extracting reviews from page 1 2019-11-21 16:21:37,150 INFO 291 :main.py(19772) - Found 10 reviews on page 1 Traceback (most recent call last): File "main.py", line 461, in main() File "main.py", line 441, in main reviews_df = extract_from_page() File "main.py", line 295, in extract_from_page data = extract_review(review) File "main.py", line 281, in extract_review res[field] = scrape(field, review, author) File "main.py", line 264, in scrape return fdictfield File "main.py", line 156, in scrape_years 'reviewBodyCell').find_element_by_tag_name('p') File "C:\Users\E20008699\AppData\Local\Continuum\anaconda3\lib\site-packages\selenium\webdriver\remote\webelement.py", line 305, in find_element_by_tag_name return self.find_element(by=By.TAG_NAME, value=name) File "C:\Users\E20008699\AppData\Local\Continuum\anaconda3\lib\site-packages\selenium\webdriver\remote\webelement.py", line 659, in find_element {"using": by, "value": value})['value'] File "C:\Users\E20008699\AppData\Local\Continuum\anaconda3\lib\site-packages\selenium\webdriver\remote\webelement.py", line 633, in _execute return self._parent.execute(command, params) File "C:\Users\E20008699\AppData\Local\Continuum\anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute self.error_handler.check_response(response) File "C:\Users\E20008699\AppData\Local\Continuum\anaconda3\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"p"} (Session info: headless chrome=78.0.3904.97)

beliakon commented 4 years ago

I was able to found similar solution in previous issue.