search

MajideND / scraping-reviews-from-googlemaps

This is a simple script with python to scrap Google Maps reviews and ratings.
MIT License
38 stars 18 forks source link

Script crashes if it's not the default address #8

Closed TheCastes closed 4 weeks ago

TheCastes commented 1 month ago

When using another address that is not the standard one in the env.py file, i encounter this error

Traceback (most recent call last): File "/home/cristianc/Scrivania/ads-scraper/app.py", line 64, in counter result = driver.find_element_by_xpath('//body/div[2]/div[3]/div[8]/div[9]/div[1]/div[1]/div[1]/div[2]/div[1]/div[1]/div[1]/div[1]/div[2]/div[1]/div[1]/div[2]').find_element_by_class_name('fontBodySmall').text ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/cristianc/Scrivania/ads-scraper/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 394, in find_element_by_xpath return self.find_element(by=By.XPATH, value=xpath) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/cristianc/Scrivania/ads-scraper/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 976, in find_element return self.execute(Command.FIND_ELEMENT, { ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/cristianc/Scrivania/ads-scraper/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute self.error_handler.check_response(response) File "/home/cristianc/Scrivania/ads-scraper/lib/python3.12/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//body/div[2]/div[3]/div[8]/div[9]/div[1]/div[1]/div[1]/div[2]/div[1]/div[1]/div[1]/div[1]/div[2]/div[1]/div[1]/div[2]"} (Session info: chrome-headless-shell=127.0.6533.99)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/cristianc/Scrivania/ads-scraper/app.py", line 119, in counter = counter() ^^^^^^^^^ File "/home/cristianc/Scrivania/ads-scraper/app.py", line 67, in counter result = driver.find_element_by_xpath('//body/div[2]/div[3]/div[8]/div[9]/div[1]/div[1]/div[1]/div[2]/div[1]/div[1]/div[1]/div[1]/div[2]/div[2]/div[1]/div[2]').find_element_by_class_name('fontBodySmall').text ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/cristianc/Scrivania/ads-scraper/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 394, in find_element_by_xpath return self.find_element(by=By.XPATH, value=xpath) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/cristianc/Scrivania/ads-scraper/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 976, in find_element return self.execute(Command.FIND_ELEMENT, { ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/cristianc/Scrivania/ads-scraper/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute self.error_handler.check_response(response) File "/home/cristianc/Scrivania/ads-scraper/lib/python3.12/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//body/div[2]/div[3]/div[8]/div[9]/div[1]/div[1]/div[1]/div[2]/div[1]/div[1]/div[1]/div[1]/div[2]/div[2]/div[1]/div[2]"} (Session info: chrome-headless-shell=127.0.6533.99)

MajideND commented 1 month ago

Can you send an example url you have used in the env file?

On Wed, 14 Aug 2024, 20:37 TheCastes, @.***> wrote:

When using another address that is not the standard one in the env.py file, i encounter this error

Traceback (most recent call last): File "/home/cristianc/Scrivania/ads-scraper/app.py", line 64, in counter result = driver.find_element_by_xpath('//body/div[2]/div[3]/div[8]/div[9]/div[1]/div[1]/div[1]/div[2]/div[1]/div[1]/div[1]/div[1]/div[2]/div[1]/div[1]/div[2]').find_element_by_class_name('fontBodySmall').text

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/cristianc/Scrivania/ads-scraper/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 394, in find_element_by_xpath return self.find_element(by=By.XPATH, value=xpath) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/cristianc/Scrivania/ads-scraper/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 976, in find_element return self.execute(Command.FIND_ELEMENT, { ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/cristianc/Scrivania/ads-scraper/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute self.error_handler.check_response(response) File "/home/cristianc/Scrivania/ads-scraper/lib/python3.12/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//body/div[2]/div[3]/div[8]/div[9]/div[1]/div[1]/div[1]/div[2]/div[1]/div[1]/div[1]/div[1]/div[2]/div[1]/div[1]/div[2]"} (Session info: chrome-headless-shell=127.0.6533.99)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/cristianc/Scrivania/ads-scraper/app.py", line 119, in counter = counter() ^^^^^^^^^ File "/home/cristianc/Scrivania/ads-scraper/app.py", line 67, in counter result = driver.find_element_by_xpath('//body/div[2]/div[3]/div[8]/div[9]/div[1]/div[1]/div[1]/div[2]/div[1]/div[1]/div[1]/div[1]/div[2]/div[2]/div[1]/div[2]').find_element_by_class_name('fontBodySmall').text

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/cristianc/Scrivania/ads-scraper/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 394, in find_element_by_xpath return self.find_element(by=By.XPATH, value=xpath) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/cristianc/Scrivania/ads-scraper/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 976, in find_element return self.execute(Command.FIND_ELEMENT, { ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/cristianc/Scrivania/ads-scraper/lib/python3.12/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute self.error_handler.check_response(response) File "/home/cristianc/Scrivania/ads-scraper/lib/python3.12/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response raise exception_class(message, screen, stacktrace) selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//body/div[2]/div[3]/div[8]/div[9]/div[1]/div[1]/div[1]/div[2]/div[1]/div[1]/div[1]/div[1]/div[2]/div[2]/div[1]/div[2]"} (Session info: chrome-headless-shell=127.0.6533.99)

— Reply to this email directly, view it on GitHub https://github.com/MajideND/scraping-reviews-from-googlemaps/issues/8, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALKUT42I45TDESDZXLV4EK3ZROFGFAVCNFSM6AAAAABMQYFWIWVHI2DSMVQWIX3LMV43ASLTON2WKOZSGQ3DMMZWHAYTINA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

TheCastes commented 1 month ago

The two links I used are these @MajideND :

https://www.google.com/maps/place//data=!4m2!3m1!1s0x4875efde7d1f391b:0x59dda4bf018973ff?entry=s&sa=X&ved=1t:8290&hl=it-it&ictx=111

https://www.google.com/maps/place/La+Dolce+Legnano,+Via+Abruzzi,+14,+20025+Legnano+MI/@45.5861246,8.9005932,17z/data=!4m6!3m5!1s0x47868d339f625a31:0x9a9ed3e95cf6ab65!8m2!3d45.5861246!4d8.9005932!16s%2Fg%2F11h0mp24r4?hl=it-it

MajideND commented 1 month ago

It would help if you used the reviews tab URL in the eny.py file. it should be: https://www.google.com/maps/place/Aeroporto+di+Londra-Gatwick/@51.1536621,-0.1820629,17z/data=!4m8!3m7!1s0x4875efde7d1f391b:0x59dda4bf018973ff!8m2!3d51.1536621!4d-0.1820629!9m1!1b1!16zL20vMHByNnM?hl=it-it&entry=ttu and https://www.google.com/maps/place/La+Dolce+Legnano/@45.5861246,8.9005932,17z/data=!4m8!3m7!1s0x47868d339f625a31:0x9a9ed3e95cf6ab65!8m2!3d45.5861246!4d8.9005932!9m1!1b1!16s%2Fg%2F11h0mp24r4?hl=it-it&entry=ttu

If it worked please let me know to close the issue.