matthewmueller / x-ray

The next web scraper. See through the <html> noise.
MIT License
5.87k stars 349 forks source link

ENOTFOUND - incorrect hostname #260

Closed xauxatz closed 5 years ago

xauxatz commented 7 years ago

ENOTFOUND - incorrect hostname

I call xray with the following URL: www.wirdgroup.com This is a correctly working URL xray returns with error ENOTFOUND and

Note the incorrect ending "comde" which obviously is not a correct URL. My theory is that www.wirdgroup.com refers to www.wirdgroup.com/de and that xray in some way gets confused about this.

Your environment

I use the newest version of xray with npm in a meteor app. I run it on Windows 10 (but it gives the same error on Ubuntu)

Steps to reproduce

Use xray on the link "www.wirdgroup.com"

Expected behaviour

The link should be correctly fetched

Actual behaviour

Tell us what happens instead.

DannyCork commented 7 years ago

I used the following code

var Xray = require('x-ray');  
var x = Xray();    

x('http://www.wirdgroup.com', 'title')(function(err, result) {
    console.log('error is ' + err);
    console.log('result is ' + result);
})

which results in

error is null
result is WIRD AG Zug/Lausanne/Genève/Lugano

navigating to https://www.wirdgroup.com/ redirects my browser to https://www.wirdgroup.com/en/index.php

I can't reproduce this. My browser is set to EN , I'm guessing yours is set to DE and hence the different redirect ? Try my code above and let me know the output.

lathropd commented 5 years ago

Closing as stale due to nonresponse to Danny.