The "url" argument must be of type string. Received undefined

nitsanasraf commented 2 years ago

I am trying to scrape a bunch of categories, and after like 3 images, I get this error and it stops the agent from scraping more.

It does work when I don't use many categories, and for some queries, it works, so overall not stable at all for some reason.

That's my code:

const Scraper = require("images-scraper");

const google = new Scraper({
  puppeteer: {
    headless: false,
  },
});

const categories = ["shark", "cat", "lion", "snake", "spider"];

(async () => {
  try {
    const results = await google.scrape(categories, 10);
    results.forEach((item, index) => {
      item.images.forEach((img, i) => {
        //do something with images
      });
    });
  } catch (e) {
    console.log("Scraping failed", e);
  }
})();