matthewmueller / x-ray

The next web scraper. See through the <html> noise.
MIT License
5.87k stars 349 forks source link

Process Multiple URLs #274

Closed geraldsamosir closed 5 years ago

geraldsamosir commented 6 years ago

Process Multiple URLs

how to use x-ray js for scraping multiple url

at this case all of that url have render to the same dom

can you give me some example ?

if have try like this

  alluri.forEach(function(uri) {
           x(uri,{
            title : '.product-title@text',
            idshop  :'#shop-id@value',
            price  : "span[itemprop='price']",
            detail : "p[itemprop='description']",
            berat  : ".detail-info@text",
            gambar  :"img[itemprop='image']@src"
        })(function(err,obj){
                obj.berat = obj.berat.replace(/\s/g,'') 
                obj.berat =  obj.berat.slice(parseInt(obj.berat.lastIndexOf("Berat") + 5) , obj.berat.lastIndexOf("Terjual"))
                console.log(obj)
         })

that's work but my problem i canot return it as json for my end point

maybe you have another way ?

lathropd commented 5 years ago

This won't return valid JSON because it's dumping each URI set separately. Also it's dumping it to console, although I'm guessing you were doing something else in production?

Did you ever find a solution? If not will reopen as a question.