Main reason why I added this was because I was generating a static build for a simple site, but for some reason it crawls a url with a different domain (localhost with different port). I would try and investigate the localhost issue, but having an exclude option will cover future problems as well.
Adding some simple logic to exclude specific URLs from crawl queue.
Did not include exclude in the defaultOptions (not sure how to properly update the snapshot 🤷).
Unit test
Feel free to close, but so far tests are good. I will use the forked version for my use case right now, but having it in the main repo would be nicer.
Description
Main reason why I added this was because I was generating a static build for a simple site, but for some reason it crawls a url with a different domain (localhost with different port). I would try and investigate the localhost issue, but having an exclude option will cover future problems as well.
excludein thedefaultOptions(not sure how to properly update the snapshot 🤷).Feel free to close, but so far tests are good. I will use the forked version for my use case right now, but having it in the main repo would be nicer.
Cheers!