nrabinowitz / pjscrape

A web-scraping framework written in Javascript, using PhantomJS and jQuery
http://nrabinowitz.github.io/pjscrape/
MIT License
997 stars 159 forks source link

Can't get the itemfile or file writers to work #24

Closed simonexmachina closed 12 years ago

simonexmachina commented 12 years ago

I've read the docs and checked the code, but the following config doesn't result in a file being written. I'm using the latest version of PhantomJS (1.6.1) and master branch of pjscrape (0.1.4 in VERSION.txt)

pjs.addSuite({
  url: 'http://www.google.com/',
  writer: 'itemfile',
  // outFile: '/tmp/pjscrape-out.txt',
  scraper: function() {
    return {
      filename: '/tmp/pjscrape-out.txt',
      content: document.documentElement.outerHTML
    }
  }
});

I've also tried 'file' writer and outFile in the top-level config, but neither results in a file being written.

nrabinowitz commented 12 years ago

Writers and loggers can't be set in .addSuite() - they need to be specified in .config():

pjs.config({
  writer: 'itemfile',
});

pjs.addSuite({
  url: 'http://www.google.com/',
  scraper: function() {
    return {
      filename: 'pjscrape-out.txt',
      content: document.documentElement.outerHTML
    }
  }
});
simonexmachina commented 12 years ago

Awesome, thanks Nick.