matthewmueller / x-ray

The next web scraper. See through the <html> noise.
MIT License
5.87k stars 349 forks source link

Does not seem to work with Node v8.0.0 #263

Closed Ilshidur closed 7 years ago

Ilshidur commented 7 years ago

Subject of the issue

The Xray module doesn't act as expected with the latest Node version.

Your environment

Steps to reproduce

Run this code using Node v8.0.0 :

x('http://www.probuilds.net/champions', '.champion-results', ['li@data-id'])((err, result) => {
  // This code won't be executed
  if (err) {
    reject(err);
    return;
  }
  resolve(result);
});

Expected behaviour

The callback is run, eventually with an err parameter is an error occurs.

Actual behaviour

The module seems to suddenly stop without throwing an error nor run the callback function, maybe due to an internal error. However, surrounding the x() function with a try { ... } catch() { ... } block won't work either, as the function does not throw any error.

DannyCork commented 7 years ago

Tried with: Node JS v8.1.3 Npm 5.0.3 x-ray 2.3.2 (2017-02-24)

The follow code contains an invalid url and generates and error as expected:

var Xray = require('x-ray');  
var x = Xray();    

x('http://www.probuilds.INVALID/champions', '.champion-results', ['li@data-id'])((err, result) => {
  if (err) {
    console.log('An error occured');
    console.log(err);
    return;
  }
  console.log('No error , here are the results ')
  console.log(result);
});
An error occured
{ Error: getaddrinfo ENOTFOUND www.probuilds.invalid www.probuilds.invalid:80
    at errnoException (dns.js:50:10)
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:91:26)
  code: 'ENOTFOUND',
  errno: 'ENOTFOUND',
  syscall: 'getaddrinfo',
  hostname: 'www.probuilds.invalid',
  host: 'www.probuilds.invalid',
  port: 80,
  response: undefined }

The following code works without any error

var Xray = require('x-ray');  
var x = Xray();    

x('http://www.probuilds.net/champions', '.champion-results', ['li@data-id'])((err, result) => {
  if (err) {
    console.log('An error occured');
    console.log(err);
    return;
  }
  console.log('No error , here are the results ')
  console.log(result);
});
No error , here are the results
[ 'Aatrox|Aatrox|Fighter,Tank',
  'Ahri|Ahri|Mage,Assassin',
  'Akali|Akali|Assassin',
  'Alistar|Alistar|Tank,Support',
  'Amumu|Amumu|Tank,Mage',
  'Anivia|Anivia|Mage,Support',
  'Annie|Annie|Mage',
  'Ashe|Ashe|Marksman,Support',
  'AurelionSol|Aurelion Sol|Mage,Fighter',
  'Azir|Azir|Mage,Marksman',
  'Bard|Bard|Support,Mage',
  'Blitzcrank|Blitzcrank|Tank,Fighter',
  'Brand|Brand|Mage',
  'Braum|Braum|Support,Tank',
  'Caitlyn|Caitlyn|Marksman',
  'Camille|Camille|Fighter,Tank',
  'Cassiopeia|Cassiopeia|Mage',
  'Chogath|Cho\'Gath|Tank,Mage',
  'Corki|Corki|Marksman',
  'Darius|Darius|Fighter,Tank',
  'Diana|Diana|Fighter,Mage',
  'DrMundo|Dr. Mundo|Fighter,Tank',
  'Draven|Draven|Marksman',
  'Ekko|Ekko|Assassin,Fighter',
  'Elise|Elise|Mage,Fighter',
  'Evelynn|Evelynn|Assassin,Mage',
  'Ezreal|Ezreal|Marksman,Mage',
  'Fiddlesticks|Fiddlesticks|Mage,Support',
  'Fiora|Fiora|Fighter,Assassin',
  'Fizz|Fizz|Assassin,Fighter',
  'Galio|Galio|Tank,Mage',
  'Gangplank|Gangplank|Fighter',
  'Garen|Garen|Fighter,Tank',
  'Gnar|Gnar|Fighter,Tank',
  'Gragas|Gragas|Fighter,Mage',
  'Graves|Graves|Marksman',
  'Hecarim|Hecarim|Fighter,Tank',
  'Heimerdinger|Heimerdinger|Mage,Support',
  'Illaoi|Illaoi|Fighter,Tank',
  'Irelia|Irelia|Fighter,Assassin',
  'Ivern|Ivern|Support,Mage',
  'Janna|Janna|Support,Mage',
  'JarvanIV|Jarvan IV|Tank,Fighter',
  'Jax|Jax|Fighter,Assassin',
  'Jayce|Jayce|Fighter,Marksman',
  'Jhin|Jhin|Marksman,Assassin',
  'Jinx|Jinx|Marksman',
  'Kalista|Kalista|Marksman',
  'Karma|Karma|Mage,Support',
  'Karthus|Karthus|Mage',
  'Kassadin|Kassadin|Assassin,Mage',
  'Katarina|Katarina|Assassin,Mage',
  'Kayle|Kayle|Fighter,Support',
  'Kennen|Kennen|Mage,Marksman',
  'Khazix|Kha\'Zix|Assassin,Fighter',
  'Kindred|Kindred|Marksman',
  'Kled|Kled|Fighter,Tank',
  'KogMaw|Kog\'Maw|Marksman,Mage',
  'Leblanc|LeBlanc|Assassin,Mage',
  'LeeSin|Lee Sin|Fighter,Assassin',
  'Leona|Leona|Tank,Support',
  'Lissandra|Lissandra|Mage',
  'Lucian|Lucian|Marksman',
  'Lulu|Lulu|Support,Mage',
  'Lux|Lux|Mage,Support',
  'Malphite|Malphite|Tank,Fighter',
  'Malzahar|Malzahar|Mage,Assassin',
  'Maokai|Maokai|Tank,Mage',
  'MasterYi|Master Yi|Assassin,Fighter',
  'MissFortune|Miss Fortune|Marksman',
  'Mordekaiser|Mordekaiser|Fighter',
  'Morgana|Morgana|Mage,Support',
  'Nami|Nami|Support,Mage',
  'Nasus|Nasus|Fighter,Tank',
  'Nautilus|Nautilus|Tank,Fighter',
  'Nidalee|Nidalee|Assassin,Fighter',
  'Nocturne|Nocturne|Assassin,Fighter',
  'Nunu|Nunu|Support,Fighter',
  'Olaf|Olaf|Fighter,Tank',
  'Orianna|Orianna|Mage,Support',
  'Pantheon|Pantheon|Fighter,Assassin',
  'Poppy|Poppy|Tank,Fighter',
  'Quinn|Quinn|Marksman,Fighter',
  'Rakan|Rakan|Support',
  'Rammus|Rammus|Tank,Fighter',
  'RekSai|Rek\'Sai|Fighter',
  'Renekton|Renekton|Fighter,Tank',
  'Rengar|Rengar|Assassin,Fighter',
  'Riven|Riven|Fighter,Assassin',
  'Rumble|Rumble|Fighter,Mage',
  'Ryze|Ryze|Mage,Fighter',
  'Sejuani|Sejuani|Tank,Fighter',
  'Shaco|Shaco|Assassin',
  'Shen|Shen|Tank,melee',
  'Shyvana|Shyvana|Fighter,Tank',
  'Singed|Singed|Tank,Fighter',
  'Sion|Sion|Tank,Fighter',
  'Sivir|Sivir|Marksman',
  'Skarner|Skarner|Fighter,Tank',
  'Sona|Sona|Support,Mage',
  ... 36 more items ]

Doesn't appear to be any issue with x-ray from what I can see.

Ilshidur commented 7 years ago

Just a note for people having the same problem as me : The issue was due to the PhantomJS package (x-ray-phantom). I was using PhantomJS version 1.9.7.

Updating PhantomJS to version 2.1.0 fixed the issue for me.