issues
search
N0taN3rd
/
Squidwarc
Squidwarc is a high fidelity, user scriptable, archival crawler that uses Chrome or Chromium with or without a head
https://n0tan3rd.github.io/Squidwarc/
Apache License 2.0
166
stars
26
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Fatal error when starting crawl
#53
nvanderperren
opened
3 years ago
1
Update conf.json to reflect working options for mode
#52
ericskiff
closed
7 months ago
0
docs: Missing comma in provided config in README causes invalid JSON example
#51
machawk1
closed
4 years ago
1
Setting mode in conf.json is ignored
#50
DerChaot
opened
5 years ago
1
Applying some raid to them bugs
#49
N0taN3rd
closed
5 years ago
0
Exception with defaults using Docker in Puppeteer config
#48
machawk1
closed
5 years ago
2
Dev + Nex -> Master
#47
N0taN3rd
closed
5 years ago
0
Incorporate next branch into dev
#46
N0taN3rd
closed
5 years ago
0
TypeError in node-warc causes Squidwarc to crash
#45
machawk1
closed
5 years ago
2
Installation fails with missing node-warc submodule
#44
peterk
closed
5 years ago
3
Error: options.stripFragment is renamed to options.stripHash
#43
ghost
closed
5 years ago
1
Feature request: following links
#42
ghost
closed
5 years ago
5
Feature request: ignore sets
#41
ghost
opened
5 years ago
0
Append config seem working not correctly.
#40
ghost
closed
5 years ago
3
Unable to reuse local Chrome user dir/cookies
#39
machawk1
opened
5 years ago
7
Describe requirements in the README
#38
machawk1
opened
5 years ago
1
Feature request: set browser accept language
#37
peterk
opened
5 years ago
6
Make option for gzipped WARC:s?
#36
peterk
closed
5 years ago
1
Fix typo in quick start within manual
#35
machawk1
closed
5 years ago
0
Add docs to instruct bootstrapping. Closes #33
#34
machawk1
closed
5 years ago
0
Exception immediately thrown on first-run
#33
machawk1
closed
5 years ago
5
Generated WARC files fail indexing with OpenWayback cdx-indexer
#32
peterk
closed
5 years ago
11
Squidwarc on Windows 10 (using npm)
#31
mw0000
opened
5 years ago
1
Documentation work
#30
N0taN3rd
closed
6 years ago
0
Squidwarc is now user scriptable via puppeteer!
#29
N0taN3rd
closed
6 years ago
0
Puppeteer crawler Test
#28
N0taN3rd
closed
6 years ago
0
Update .gitmodules
#27
peterk
closed
6 years ago
1
Instagram pages not recorded properly?
#26
peterk
closed
5 years ago
3
Allow crawl config to contain user scripts for a page
#25
peterk
closed
5 years ago
16
Add docker file
#24
N0taN3rd
closed
5 years ago
0
TypeError: input.on is not a function
#23
peterk
closed
5 years ago
6
Merge node-warc version bump that will fix issue of bad warc generation.........
#22
N0taN3rd
closed
6 years ago
0
add support for one big warc file
#21
raspher
closed
5 years ago
4
Syntax error with awk usage on Mac
#20
machawk1
closed
5 years ago
1
Use bluebird's promise and not the native version on Mac.
#19
machawk1
closed
6 years ago
1
Support a simple list of URIs
#18
machawk1
closed
5 years ago
3
Make scalable
#17
N0taN3rd
opened
6 years ago
2
Merge from Dev to Master 1.1.0
#16
N0taN3rd
closed
6 years ago
0
Add logo + src
#15
machawk1
closed
6 years ago
1
Original Response headers (i.e., start with X-Archive-Orig-...) are modified
#14
maturban
opened
7 years ago
1
Recursion
#13
zsau
closed
5 years ago
3
Add git submodule nodewarc
#12
N0taN3rd
closed
7 years ago
0
Add node-warc as a sub-module
#11
N0taN3rd
closed
7 years ago
0
Enable userland choice of keeping HTTP/2 or modifying HTTP/2 -> HTTP/…
#10
N0taN3rd
closed
7 years ago
0
Enable userland choice of keeping HTTP/2 or turning it into HTTP/1.1
#9
N0taN3rd
closed
7 years ago
0
Minor correction: "heretrix" is female
#8
mhucka
closed
7 years ago
1
Critical patch fixes #6
#7
N0taN3rd
closed
7 years ago
0
Infinite loop in crawling
#6
N0taN3rd
closed
7 years ago
0
Changed default mode and mod check that was from previous revision
#5
N0taN3rd
closed
7 years ago
0
configRunner is defaulting to page-all-links
#4
N0taN3rd
closed
7 years ago
0
Next