search

let4be / crusty

Broad Web Crawler
GNU General Public License v3.0
83 stars 3 forks source link

Add WARC support #1

Open let4be opened 3 years ago

let4be commented 3 years ago

seems like https://github.com/jedireza/warc could help