search

tether / roach

A very adaptable web crawler framework. Impossible to kill.
Other
7 stars 1 forks source link

Crawlers should support parsing Text #12

Closed ekryski closed 10 years ago

bredele commented 10 years ago

done with a little bonus:

    crawler('file://' + __dirname + '/fixtures/roach.text')
      .pipe(crawler.split())
      .pipe(crawler.mapSync(function(data) {
        return data.toUpperCase();
      }))
      .pipe(crawler.join(' '))
      .pipe(crawler.wait())
      .on('data', function(data) {
         //return transformed string
      });