Getting every type of url from the page source

amoilanen / js-crawler

Web crawler for Node.JS

MIT License

253 stars 55 forks source link

Getting every type of url from the page source #25

Open shekarls opened 8 years ago

shekarls commented 8 years ago

Hello, Thanks for the nice robust crawler, Currently js-crawler crawls thru all visible links on the page, but I was wondering if it supports other href types like img/rel/src etc to be scrapped from the page.

Thanks shekar

amoilanen commented 8 years ago

Thank you for using the crawler and for a feature suggestion. In the next version of crawler we may consider including options like crawlImg, crawlRelSrc which will enable crawler to crawl such links.