search

gaspa93 / googlemaps-scraper

Google Maps reviews scraping
GNU General Public License v3.0
394 stars 145 forks source link

Stale Element Reference Error #36

Open Maunil2k opened 1 year ago

Maunil2k commented 1 year ago

Hi @gaspa93, I was trying to use the following URL: https://www.google.com/maps/place/Ellora+Caves/@20.025817,75.1779975,17z/data=!4m7!3m6!1s0x3bdb93bd138ae4bd:0x574c6482cf0b89cf!8m2!3d20.025817!4d75.1779975!9m1!1b1 for scraping N=1000 reviews and sort by = most relevant when I got this error: selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document Full error:

[Review 0] Traceback (most recent call last): File "scraper.py", line 63, in reviews = scraper.get_reviews(n) File "/home/maunil/Desktop/googlemaps-scraper/googlemaps.py", line 172, in get_reviews self.__expand_reviews() File "/home/maunil/Desktop/googlemaps-scraper/googlemaps.py", line 298, in __expand_reviews l.click() File "/home/maunil/Desktop/venv/lib/python3.8/site-packages/selenium/webdriver/remote/webelement.py", line 80, in click self._execute(Command.CLICK_ELEMENT) File "/home/maunil/Desktop/venv/lib/python3.8/site-packages/selenium/webdriver/remote/webelement.py", line 628, in _execute return self._parent.execute(command, params) File "/home/maunil/Desktop/venv/lib/python3.8/site-packages/selenium/webdriver/remote/webdriver.py", line 320, in execute self.error_handler.check_response(response) File "/home/maunil/Desktop/venv/lib/python3.8/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document (Session info: headless chrome=103.0.5060.114)

gaspa93 commented 1 year ago

Hi @Maunil2k, I tried the link provided and it seems to work. What is the exact command you are trying to execute?

Maunil2k commented 1 year ago

Hi @gaspa93 , This is command that I'm using python scraper.py --N 1000 --sort_by most_relevant

gaspa93 commented 1 year ago

I tried with lower value of N and it works correctly: do you get the error immediately or after some time?

Maunil2k commented 1 year ago

I get the error almost 2 secs after the crawler selects most relevant from the sort by menu.

gaspa93 commented 1 year ago

Very strange, it works fine for me: did you have all requirements installed and updated?