Open TheTechRobo opened 3 years ago
I started a crawl, with the default settings. However, I only have 3GB free, and there are 100k URLs in the queue. I need to change the warc-max-size, and the finished-warc-dir options. How can I do this mid-crawl, if it's possible? Thanks!
I started a crawl, with the default settings. However, I only have 3GB free, and there are 100k URLs in the queue. I need to change the warc-max-size, and the finished-warc-dir options. How can I do this mid-crawl, if it's possible? Thanks!