search

ukwa / ukwa-heritrix

The UKWA Heritrix3 custom modules and Docker builder.
10 stars 7 forks source link

WARCViralWriterProcessor and revisits? #3

Closed PsypherPunk closed 9 years ago

PsypherPunk commented 10 years ago

Currently we're skipping writing revisit records with the WARCViralWriterProcessor (i.e. writeRevisitForIdenticalDigests is false).

We need to verify that this kludge can actually handle revisits correctly.

anjackson commented 9 years ago

This is not worth the additional complexity, for such a small fraction of the content. Won't-fix for now.