matthewmueller / x-ray

The next web scraper. See through the <html> noise.
MIT License
5.87k stars 349 forks source link

Errors get swallowed when nested selectors are used #203

Open jspri opened 8 years ago

jspri commented 8 years ago

Subject of the issue

Errors that are thrown within the xray callback are suppressed when there is a nested selector.

Your environment

The following script has no output when run. You would expect an error message/stack trace to appear.

const Xray = require('x-ray')
const x = Xray()

const map = {
    title: 'title',
    links: x('a', ['@href'])
}

x('http://www.google.com', map)(function(err, res) {
    throw(Error("this is an error"))
})
jspri commented 8 years ago

A bit more investigation show that if the code is changed to the following the error will be shown. This suggests that the callback is being called twice, once to handle the result set and once to handle the error.

x('http://www.google.com', map)(function(err, res) {
    if(err) {
        console.log(err)
    }

    throw(Error("this is an error"))
})

Alternatively if a user were to do something like this...

function scrapeSomething(callback) {
    x('http://www.google.com', map)(function(err, res) {
        if(err) {
            callback(err)
        }

        throw(Error("this is an error"))
    })
}

The more idiomatic node code in the second example would result in the error bubbling up to closer match what the expected behaviour would be. Likely making the bug less likely to surface in many situations.

jspri commented 8 years ago

Possibly related to #135/#181. Reopened as x-ray 2.3.1 did not fix.

jspri commented 7 years ago

Found to be issue with batch library https://github.com/visionmedia/batch/issues/32