how does/will this data get supplied to was-registrar
I believe we should hold off on this until we have was-downloader writing warc files to a directory (that will be organized by crawl id, if I have my way).
I'm also wondering if this should be triggered via cron job on was-registrar VM, with code associated with was-registrar (as this is a manifest created solely for was-registrar)
I believe we should hold off on this until we have was-downloader writing warc files to a directory (that will be organized by crawl id, if I have my way).
I'm also wondering if this should be triggered via cron job on was-registrar VM, with code associated with was-registrar (as this is a manifest created solely for was-registrar)