issues
search
eklem
/
browsercrawler
Crawling content from a site within the browser. A basis for i.e. a search solution for static sites.
https://eklem.github.io/browsercrawler/doc/
MIT License
2
stars
0
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Make it work with browserified search-index
#16
eklem
closed
6 years ago
2
Clear excess whitespace and lineshift (lf/cr)
#15
eklem
opened
6 years ago
1
Wrap it in a function so it can be run when needed
#14
eklem
closed
6 years ago
1
Take an array of url's and crawl
#13
eklem
opened
6 years ago
2
Convert selected content to object
#12
eklem
closed
6 years ago
1
selectors config file
#11
eklem
opened
6 years ago
3
Gather URLs to crawl
#10
eklem
opened
6 years ago
2
Obey robots.txt
#9
eklem
closed
6 years ago
1
Add node stream functionality (browserified)
#8
eklem
closed
6 years ago
2
Get some tests going
#7
eklem
opened
6 years ago
0
test case
#6
eklem
opened
6 years ago
0
Test out browserifying request
#5
eklem
closed
6 years ago
1
Test it on github.io-pages
#4
eklem
closed
6 years ago
1
Pattern for where the crawler is allowed
#3
eklem
opened
6 years ago
1
Fetch url's on pages
#2
eklem
opened
6 years ago
0
Browserified request + cheerio
#1
eklem
closed
6 years ago
8
Previous