search

NikolaiT / GoogleScraper

A Python module to scrape several search engines (like Google, Yandex, Bing, Duckduckgo, ...). Including asynchronous networking support.
https://scrapeulous.com/
Apache License 2.0
2.6k stars 734 forks source link

Scraping answer boxes #238

Open danyaljj opened 4 years ago

danyaljj commented 4 years ago

Wondering if GoogleScraper has the functionality to scrape Google answer boxes?

Here is an example:

Screen Shot 2020-06-26 at 6 03 21 PM
Himanxhu commented 2 years ago

GoogleScraper --config-file config.py -m selenium --sel-browser chrome --browser-mode ˓→normal --keyword-file list.txt -o results.json -z10

This will start 10 browser windows that begin to scrape the keywords in the provided file. After about 22 minutes of scraping, I got the following results in a json file. As you can see there are 1000 results with 10 results per page and all links and snippets and stuff. You can now analize this data and make marketing decision based on it.