search

ContentMine / quickscrape

A scraping command line tool for the modern web
MIT License
259 stars 42 forks source link

same short arg for urllist and ratelimit -r #53

Closed chreman closed 9 years ago

chreman commented 9 years ago

quickscrape -r fulltext_html_urls.txt -o output/ -d ../journal-scrapers/scrapers/ -r 5 (works with -r urls.txt --ratelimit 5, but not with --urllist urls.txt -r 5)

fs.js:427 return binding.open(pathModule._makeLong(path), stringToFlags(flags), mode); ^ Error: ENOENT, no such file or directory '10' at Object.fs.openSync (fs.js:427:18) at Object.fs.readFileSync (fs.js:284:15) at loadUrls (/home/workshop/.nvm/v0.10.24/lib/node_modules/quickscrape/bin/quickscrape.js:105:17) at Object. (/home/workshop/.nvm/v0.10.24/lib/node_modules/quickscrape/bin/quickscrape.js:115:41) at Module._compile (module.js:456:26) at Object.Module._extensions..js (module.js:474:10) at Module.load (module.js:356:32) at Function.Module._load (module.js:312:12) at Function.Module.runMain (module.js:497:10) at startup (node.js:119:16)

blahah commented 9 years ago

Thanks - the short form for --ratelimit is now -i