search

phantombuster / nickjs

Web scraping library made by the Phantombuster team. Modern, simple & works on all websites. (Deprecated)
https://nickjs.org
ISC License
500 stars 48 forks source link

Found a bug when I crawl on google map #41

Closed msaus closed 6 years ago

msaus commented 6 years ago

Hello there,

Possibly, this is bug on headless chrome, but, I want to confirm if this is a bug.

When I crawl like following on google map, the result is less than I use just chrome browser.

  1. Open this URL. https://www.google.co.jp/maps/@35.6603142,139.7055783,21z

  2. sendkey like shibuya web consultant

The result of google chrome has more than 20 search results.

On the other hand, the result of chrome headless browser has aorund 8 search results.

Do you have the same result?

My environment is as following.

ubuntu 18.04 Google Chrome 67.0.3396.99 NickJs the latest one Regards,

msaus commented 6 years ago

possibly, this is related to geolocation. Is there any way to change geolocation?

paps commented 6 years ago

possibly, this is related to geolocation

It might, yes. Can you try using setGeolocationOverride()?

These low-level methods from the DevTools protocol are all available through NickJS' Chrome Remote Interface instance like so: await tab.driver.client.Page.setGeolocationOverride(...)

msaus commented 6 years ago

Thanks for your comment. I just tried that, but, still less than 10. Possibly, this is a bug for headless chrome. I just reported this to google and when they replied I let you know. Thanks :)

msaus commented 6 years ago

People from Google told me that this is a bug with Google Maps. So, I will close this issue. Thanks.

paps commented 6 years ago

Thanks, can you link to your issue for clarity? :)