search

gaspa93 / googlemaps-scraper

Google Maps reviews scraping
GNU General Public License v3.0
358 stars 131 forks source link

Maximum call stack size exceeded #47

Closed stajdini closed 2 months ago

stajdini commented 11 months ago

When I set --N to 10000, it scraped 1140 reviews, then threw this message and stopped: [0724/211919.550:INFO:CONSOLE(1550)] "Uncaught RangeError: Maximum call stack size exceeded", source: /maps/_/js/k=maps.m.en.b7ZwJWQZkHM.2019.O/ck=maps.m.HgR1ySVFXik.L.W.O/m=vwr,vd,a,nrw,owc,ob2,sp,en,smi,sc,vlg,smr,as,wrc/am=BoDCIhAB/rt=j/d=1/rs=ACT90oEnDViSjerMr5DSozguPqRfPvO2Xg/ed=1/exm=sc2,per,mo,lp,ti,ds,stx,dwi,enr,pwd,dw,ppl,log,std,b (1550)

This is the url I used: https://www.google.it/maps/place/Pantheon/@41.8986108,12.4768729,17z/data=!4m18!1m9!3m8!1s0x132f604f678640a9:0xcad165fa2036ce2c!2sPantheon!8m2!3d41.8986108!4d12.4768729!9m1!1b1!16zL20vMDF4emR6!3m7!1s0x132f604f678640a9:0xcad165fa2036ce2c!8m2!3d41.8986108!4d12.4768729!9m1!1b1!16zL20vMDF4emR6?entry=ttu

Any advice would be appreciated.

stajdini commented 11 months ago

So, to answer my own question, I have noticed that this same thing happens even if I manually open the link and scroll all the way down. So, maybe it's just Google Maps not wanting to load more than a certain percentage of the total number of reviews.

gaspa93 commented 2 months ago

Ciao @stajdini ,

yes it is correct, Google have some limitations and you cannot go before a certain number of reviews. To avoid this problem, have a look at the "Monitoring functionality" section: if you keep run the scraper multiple times per day/week/month, you can obtain all the reviews from a certain point onward.

Bye, Mattia