s-rah / onionscan

OnionScan is a free and open source tool for investigating the Dark Web.
https://twitter.com/OnionScan
Other
2.89k stars 600 forks source link

Support WARC Output for Onions #87

Open s-rah opened 8 years ago

s-rah commented 8 years ago

It would be good to support dumping a site from the database in WARC format i.e. https://www.iso.org/obp/ui/#iso:std:iso:28500:ed-1:v1:en

There are a bunch of go-lang libraries in various stats of repair for dealing with the WARC format, we should probably first evaluate their suitability.