qri-io walk issues - Githubissues

qri-io / walk

Webcrawler/sitemapper

GNU General Public License v3.0

6 stars 2 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Seed domains from sitemaps in robots.txt

#30 Mr0grog opened 5 years ago
0
docs: Redrafts readme to capture vision

#29 Frijol closed 4 years ago
1
feat: orient walk around job/coordinator server paradigm

#28 b5 closed 4 years ago
4
Fix spelling in README

#27 machawk1 closed 5 years ago
1
Ensure GOPATH and reduce repeated commands

#26 Mr0grog closed 5 years ago
1
Accept invalid SSL certificates but log that they were invalid

#25 Mr0grog opened 5 years ago
2
Update readme to include build instructions

#24 lightandluck closed 5 years ago
1
build-from-source instructions shoulde be in README.md

#23 b5 closed 5 years ago
0
Recognize URLs that should not be crawled

#22 Mr0grog opened 5 years ago
0
Learning from Data.gov?

#21 Frijol closed 5 years ago
2
feat: initial API, round out flow for local proof-of-concept

#20 b5 closed 5 years ago
4
Make Badger configuration easier

#19 Mr0grog opened 5 years ago
1
Test that file actually got written in TestCBORResourceFileWriter

#18 Mr0grog opened 5 years ago
0
Clear out commented old code in `NewBadgerConfig`

#17 Mr0grog closed 5 years ago
0
Spec out Ideal interface between Scanner & Walk

#16 b5 opened 5 years ago
7
typo fixes

#15 Frijol closed 5 years ago
1
Coordinator doesn’t always automatically end when all work is done

#14 Mr0grog opened 5 years ago
0
WIP: Sitemap

#13 b5 closed 5 years ago
15
Sitemap Resource Handler

#12 b5 opened 5 years ago
0
Add a test that confirms walk stays within its configured domains/hosts/origins

#11 b5 opened 5 years ago
0
adding initial roadmap document

#10 b5 closed 5 years ago
5
Refactor Redirect Handling & Add Tests

#9 b5 opened 5 years ago
0
Improve ResourceHandler / Coordinator Communication to handle Error States & Resource Retries

#8 b5 opened 5 years ago
0
Implement Queue.Pop acknowledgement/confirmation for guaranteed delivery

#7 b5 opened 5 years ago
1
Define this project's objective & first milestone

#6 b5 closed 5 years ago
6
bring sentry's tests into this project

#5 b5 opened 5 years ago
0
Standardizing a Test Suite For External Use

#4 b5 closed 5 years ago
3
Add HTML Test Cases with resources, links & subresources

#3 b5 opened 5 years ago
0
feat: overhaul to new architecture

#2 b5 closed 5 years ago
5
docs: Add system diagram from brainstorm

#1 Mr0grog closed 5 years ago
1