thibauts / node-google-search-scraper

Google search scraper with captcha solving support
MIT License
89 stars 40 forks source link

Do something when full search is done #11

Open DBuit opened 6 years ago

DBuit commented 6 years ago

Hello,

I'm searching google voor all indexed pages of a website, and put them in an array. When it's done i wanna do something with it.

Is there a way to get a callback when the search is all done? At this moment the script just quits when the search is done.

Thank you.

rohanleach4 commented 5 years ago

A rudimentary way could be this:

var scraper = require("google-search-scraper"); var options = { query: "pharmaceutical company", limit: 10 }; function nextStep() { console.log("finished pushing to array, let's do something else"); } let urls = [];

scraper.search(options, (err, url) => { if (err) throw err; if (url) { urls.push(url); } if (urls.length === options.limit) { console.log(urls); nextStep(); } });

felangga commented 5 years ago

@rohanleach4 what if the search result is below limit ?

rohanleach4 commented 5 years ago

@felangga Sorry for the tardiness of reply. You could add another condition to it maybe?

if (urls.length <= options.limit) { console.log("Not enough info here"); }