MiLk / UniCrawler

Web crawler in Node.js
8 stars 4 forks source link

exclude link where body or title doesn't match filter from export #2

Open MiLk opened 11 years ago

MiLk commented 11 years ago

Body and title filters are done when scrapping. So the url is already linked with source page. We have to exclude it from result.