tether / roach

A very adaptable web crawler framework. Impossible to kill.
Other
7 stars 1 forks source link

Crawlers should support parsing ZIP files #15

Closed ekryski closed 10 years ago

bredele commented 10 years ago

Here's how you can unzip a file with the crawler:

 crawler('file://' + __dirname + '/fixtures/test.zip')
      .pipe(crawler.unzip())
      .on('entry', function(entry) {
        entry.pipe(crawler.through(function(data) {
          //so something on data buffer
        }));
      });