search

Monibrand / google-html-parser

Parse search engine HTML to retrieve ads and other stuff, support Google and Bing
8 stars 3 forks source link

Search results and ads parsing not working #5

Closed pankajjha closed 4 years ago

pankajjha commented 4 years ago

Hi There,

First of all really thanks for all the efforts you have made into this package, really appreciate it.

I am facing some issues with the package lately, sometimes it only parses the shopping ads, some times only single adword ads, but most importantly it is not parsing any results in the json. I searched multiple keywords like iphone, alibaba, shein, anntaylor None of them got search results in the JSON. https://share.getcloudapp.com/KouBp5Z4 https://share.getcloudapp.com/xQuDymdP https://share.getcloudapp.com/5zuXQkWo

Also, can you tell me where can I start to fixing up the parser, because google is likely to change its results page at some intervals, some pointers to fix the parser and keep it up to date will be great.

HugoPoi commented 4 years ago

In the test/mocks folder we have sample test pages, if you can provide the google html page causing the parsing issue. You can follow the guide in CONTRIBUTING.md if you want to help with a PR :smiley: Or if you have the keywords and user agent you are using for capturing, so i can reproduce.

pankajjha commented 4 years ago

carloan.txt

Here's an example Keyword - car loan Country - US uule - W+CAIQICIaQXVzdGluLFRleGFzLFVuaXRlZCBTdGF0ZXM location - Austin, Texas, United States Useragent - Unknown (as I am using some proxy crawlers to load SERP results and save it)

HugoPoi commented 4 years ago

I successfully reproduce the issue on our production setup, I will fix it today, google introduce a new template for organic results

HugoPoi commented 4 years ago

Publish the fix under v0.3.0 on npm