So the idea is basically to "replicate" the excellent Heritrix3 web UI.
We want to give a way to start, stop, pause, unpause the crawl, but also inject seeds, search in crawl logs, maybe remove stuff matching a query from the frontier.. The possibilities are endless.
So the idea is basically to "replicate" the excellent Heritrix3 web UI.
We want to give a way to start, stop, pause, unpause the crawl, but also inject seeds, search in crawl logs, maybe remove stuff matching a query from the frontier.. The possibilities are endless.