matthewmueller / x-ray

The next web scraper. See through the <html> noise.
MIT License
5.87k stars 349 forks source link

Method chaining error with .write() and .then() #279

Closed alexadusei closed 6 years ago

alexadusei commented 6 years ago

I looked through the documentation and older issues to see if this was addressed, but doesn't seem to be the case.

The documentation states you can have promise-based behaviour with method-chaining as so:

x('https://dribbble.com', 'li.group', [{
  title: '.dribbble-img strong',
  image: '.dribbble-img [data-src]@data-src',
}])
  .paginate('.next_page@href')
  .limit(3)
  .then(function (res) {
    console.log(res[0]) // prints first result
  })
  .catch(function (err) {
    console.log(err) // handle error in promise
  })

Looks good, until you add a .write() in there, like any other method.

x('https://dribbble.com', 'li.group', [{
  title: '.dribbble-img strong',
  image: '.dribbble-img [data-src]@data-src',
}])
  .paginate('.next_page@href')
  .limit(3)
  .write('results.json')
  .then(function (res) {
    console.log(res[0]) // prints first result
  })
  .catch(function (err) {
    console.log(err) // handle error in promise
  })

This gives the following error: TypeError: xray(...).paginate(...).limit(...).write(...).then is not a function. Any reason why this is?

paulxuca commented 6 years ago

Looks like .write returns the write stream, so .then won't exist on that.

alexadusei commented 6 years ago

That's a shame. Wonder if there'll any workarounds or alternatives then.

paulxuca commented 6 years ago

You can turn the stream into a promise by resolving on the done event and rejecting on error, which makes it possible to chain promises.

alexadusei commented 6 years ago

Good suggestion - thanks @paulxuca!