search

eklem / browsercrawler

Crawling content from a site within the browser. A basis for i.e. a search solution for static sites.
https://eklem.github.io/browsercrawler/doc/
MIT License
2 stars 0 forks source link

search-index-housekeeper may need to be a part of crawler-in-browser #38

Closed eklem closed 5 years ago

eklem commented 6 years ago

Figure out where the crawl queue should live. Try to make crawler-in-browser as small as possible.

eklem commented 6 years ago

Could be that I should have a function like doSomethingWith(data) with a keepTrackOfIt callback, where the doSomethingWith() could send the data to a search engine, while the keepTrackOfIt logs this in a indexedDB.