search

nasa-jpl-memex / weapons

MEMEX Weapons Pilot for the illegal weapons domain.
http://memex.jpl.nasa.gov
Apache License 2.0
14 stars 14 forks source link

Update wrangler_cron #12

Closed harsham05 closed 8 years ago

harsham05 commented 8 years ago

TODO: Add second pass of parser-indexer (to run through nutch segments) to wrangler_cron.sh to ingest atomic updates of crawl metadata lost during File Dump

harsham05 commented 8 years ago

Echo incremental status updates of the cron job

refactor MikeJ dump scripts, only want full_dump

harsham05 commented 8 years ago

fixed in https://github.com/memex-explorer/weapons/pull/15