search

pythonhacker / harvestman-crawler

Automatically exported from code.google.com/p/harvestman-crawler
1 stars 3 forks source link

Implement -nd option of wget #32

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
Hi Anand First of all thanks a lot for a nice crawler.

I am wondering if it is possible to make harvest man save all the URLs in a
single directory i.e. can I avoid the replication of directory structure ?
Something like -nd option in wget.

I am using Harvestman version 1.4.6 (One that's in Ubuntu karmic)

Thanks, Alok

Original issue reported on code.google.com by alo...@gmail.com on 8 Apr 2010 at 11:17