Add option to pass configuration to simplecrawler directly

fffunction / backstop-crawl

🕷 Crawls a site to generate a backstopjs config file with scenarios pre-populated

38 stars 18 forks source link

Add option to pass configuration to simplecrawler directly #29

Open minorOffense opened 5 years ago

minorOffense commented 5 years ago

There are a lot of options in simplecrawler. Might be useful to allow passing options directly to the crawler for really custom setups. Like limiting crawl depth or number of pages to crawl, etc.

aaronbauman commented 4 years ago

+1 I had a similar thought when writing the patch for maxDepth https://github.com/fffunction/backstop-crawl/pull/34