website-scraper / node-website-scraper

Download website to local directory (including all css, images, js, etc.)
https://www.npmjs.org/package/website-scraper
MIT License
1.55k stars 274 forks source link

Add option to suppress error when directory exists #333

Closed iFwu closed 5 years ago

iFwu commented 5 years ago

Description

There're some situations that I want to scrape multiple organized pages to one directory, but it will throw an error. So, is possible to add an option to suppress the error when directory exists(though I can delete the code of throwing error)? PS: The reason why I don't use [url1, url2, ...] in urls option is that I want fully control of the scrapping process, like use 'promise-limit' to limit the concurrency and resume scraping after break.

s0ph1e commented 5 years ago

You can use resourceSaver option to overwrite default behavior and save to existing directory.

But be aware that saving to existing directory may include next tricky things: