crawler needs to add id = URL

eklem / browsercrawler

Crawling content from a site within the browser. A basis for i.e. a search solution for static sites.

https://eklem.github.io/browsercrawler/doc/

MIT License

2 stars 0 forks source link

crawler needs to add id = URL #30

Closed eklem closed 6 years ago

eklem commented 6 years ago

Crawler() in bundle.js needs to add id: 'URL' to the item-object. This to not get duplicates. Title, body and URL is a good minimum for most use cases.