matthewmueller / x-ray

The next web scraper. See through the <html> noise.
MIT License
5.87k stars 349 forks source link

Not getting anything back? #266

Closed DilionsCode closed 5 years ago

DilionsCode commented 7 years ago

I'm trying to get #fl-carousel > div > ul > li from HomeFirstRealtors

it's the carousel on the page with houses listed that I'm trying to get

I wrote this back in the past but I never understood ever why it's not returning something, originally it worked fine now it doesn't add anything to results.json it just returns []

I'm using the newest version of x-ray and experimented with older versions and nothing changed.

Here's the code

var Xray = require('x-ray');
var x = Xray();

x('https://homesforsale.century21.com/century-21-home-first-realtors-6309c', '#fl-carousel > div > ul > li', [{
  img: 'div > div:nth-child(1) > a > img@src',
  url: 'div > div:nth-child(1) > a@href',
  ad1: '#address1',
  ad2: '#address2',
  price: '#price',
  bed: '#beds',
  bath: '#baths',
  sqft: '#sqft_price',
}])
.write('results.json')
cpheimbach commented 7 years ago

I just had a quick look and maybe found your issue. When looking up the img, you are still 'one level to high up'. You need to add another div:nth-child(1) after the one you already got there.

div > div:nth-child(1) > div:nth-child(1) > a > img@src While checking your problem I found a handy way to test your selectors: Just hit CMD+F (or windows equivalent) in your Chrome Dev tools and it let's you search the DOM. You can input the same CSS selector query and test whether you get to the elements you are after.

You just can't test the iteration but I trust your imagination skills. I hope that helps. :neckbeard: