Nodejs web scraper. Contains a command line, docker container, terraform module and ansible roles for distributed cloud scraping. Supported databases: SQLite, MySQL, PostgreSQL. Supported headless clients: Puppeteer, Playwright, Cheerio, JSdom.
107
stars
16
forks
source link
when creating a project accept multiple starting urls #35
Closed
a1sabau closed 3 years ago
right now a scraping definition can contain a single starting url with depth 0