Make WARC files searchable

ArchiveTeam / grab-site

The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns

Other

1.31k stars 129 forks source link

Make WARC files searchable #166

Open Svekla opened 4 years ago

Svekla commented 4 years ago

Sorry for asking a question not related to grab-site, but I don't really know where should I ask it.

I archived a big forum that recently went down. Unfortunately without the search function it's a pain in the butt to find something specific. Is there any way to index WARC files and make them searchable? Thank you very much for help.

brandongalbraith commented 4 years ago

You might check out https://www.archiveteam.org/index.php?title=The_WARC_Ecosystem for tooling that meets your requirements.