raulr / google-play-scraper

A PHP scraper to get app data from Google Play
MIT License
125 stars 65 forks source link

The current node list is empty #3

Closed ghost closed 8 years ago

ghost commented 8 years ago

I'm having troubles getting the data for this app:

https://play.google.com/store/apps/details?id=org.zooper.zwpro

The funny thing is, that it works fine with the free version of the same app:

https://play.google.com/store/apps/details?id=org.zooper.zwfree

I have no idea, what the problem is. Any help is appreciated.

raulr commented 8 years ago

I can't reproduce it. Can you provide more information:

ghost commented 8 years ago

Hi,

here's the full exception:

InvalidArgumentException Object
(
    [message:protected] => The current node list is empty.
    [string:Exception:private] => 
    [code:protected] => 0
    [file:protected] => /vendor/symfony/dom-crawler/Crawler.php
    [line:protected] => 547
    [trace:Exception:private] => Array
        (
            [0] => Array
                (
                    [file] => /vendor/raulr/google-play-scraper/src/Scraper.php
                    [line] => 146
                    [function] => text
                    [class] => Symfony\Component\DomCrawler\Crawler
                    [type] => ->
                    [args] => Array
                        (
                        )

                )

            [1] => Array
                (
                    [file] => /api.php
                    [line] => 60
                    [function] => getApp
                    [class] => Raulr\GooglePlayScraper\Scraper
                    [type] => ->
                    [args] => Array
                        (
                            [0] => org.zooper.zwpro
                            [1] => de
                            [2] => de
                        )

                )

            [2] => Array
                (
                    [file] => /details.php
                    [line] => 3
                    [args] => Array
                        (
                            [0] => /api.php
                        )

                    [function] => require
                )

        )

    [previous:Exception:private] => 
)

I'm currently running PHP 7.0.3. But also fails on 5.6.18.

            $scraper = new Scraper();
            $app = $scraper->getApp($id, $hl, $gl);

All other apps are working fine, except for this one.

raulr commented 8 years ago

Hi Whoknowsit,

I've been debugging this. It seems that at some point the HTML response for that app prints a NULL character, so the parsing of the document done by the DomCrawler component is interrupted and causes the error.

I've made a pull request with a fix to the Symfony repo symfony/symfony#18454

ghost commented 8 years ago

Thank you, that seems to fix it :)

raulr commented 8 years ago

Added a workaround for this bug in release 0.1.1.

ghost commented 6 years ago

Have the same problem with the actual version.

        $scraper = new Scraper();

        $app = $scraper->getApp('mm.grindulator_premium')

InvalidArgumentException : The current node list is empty. /home/markus/workspace/IALaravel/vendor/symfony/dom-crawler/Crawler.php:565 /home/markus/workspace/IALaravel/vendor/raulr/google-play-scraper/src/Scraper.php:114 /home/markus/workspace/IALaravel/tests/Feature/AppKiosk/ImportTest.php:68

Sometimes it works and sometimes I get the issue mentioned above.

jkmbo commented 6 years ago

Experienced the same. it happens on many apps now. scraper error: The current node list is empty. net.apphunter.android

scraper error: The current node list is empty. de.mcoins.applike

scraper error: The current node list is empty. com.ionicframework.wagandroid554504

scraper error: The current node list is empty. com.bigfishgames.fairwaysolitaireuniversalf2pgoogle

scraper error: The current node list is empty. com.bigfishgames.gummydropgoogle

scraper error: The current node list is empty. com.igg.android.lordsmobile

scraper error: The current node list is empty. net.lovoo.android

scraper error: The current node list is empty. com.lucktastic.scratch

scraper error: The current node list is empty. com.surveysampling.mobile.quickthoughts

scraper error: Requested resource not found com.rdm.rainydayrewards

scraper error: The current node list is empty. com.pixonic.wwr

scraper error: The current node list is empty. com.totaltrivia

scraper error: The current node list is empty. net.skoobe.reader

scraper error: The current node list is empty. com.cashtime

scraper error: The current node list is empty. com.nexters.herowars

scraper error: The current node list is empty. me.goldesel

scraper error: The current node list is empty. ru.crazybit.experiment

scraper error: The current node list is empty. com.appkarma.app

scraper error: The current node list is empty. com.gramgames.mergedragons

scraper error: The current node list is empty. com.digital.android.ilove

scraper error: The current node list is empty. com.franprix.catalogue

scraper error: The current node list is empty. com.igg.castleclash

scraper error: Requested resource not found com.pyrsoftware.pokerstars.fr

scraper error: The current node list is empty. com.bigfishgames.cookingcrazegooglef2p

scraper error: The current node list is empty. com.perblue.portalquest

scraper error: The current node list is empty. com.zynga.livepoker

scraper error: The current node list is empty. com.pantaflix.client

scraper error: The current node list is empty. com.mentormate.android.inboxdollars

scraper error: The current node list is empty. fr.casino.fidelite

scraper error: The current node list is empty. net.appbounty.android

scraper error: The current node list is empty. com.libertex.mobile

scraper error: The current node list is empty. de.yellostrom.incontrol

scraper error: Requested resource not found uk.fulltilt.fulltilt

scraper error: The current node list is empty. com.yamyam.jemieoliverrecipes

scraper error: The current node list is empty. com.ekkorr.endlessfrontier.global

scraper error: Requested resource not found uk.pokerstars.casino

scraper error: The current node list is empty. com.shoclef

scraper error: The current node list is empty. com.att.dataperks

scraper error: The current node list is empty. com.cryptohash.cryptofolio

scraper error: Requested resource not found com.pokerstars.uk

scraper error: The current node list is empty. com.geniogames.geniogames

scraper error: The current node list is empty. com.tapslots.sevenkings

scraper error: Requested resource not found com.garbagepenguin

scraper error: The current node list is empty. com.gi8wcm.clean

scraper error: The current node list is empty. com.prodege.swagiq

scraper error: The current node list is empty. com.shiku.job.push

scraper error: The current node list is empty. com.comfourk.app

scraper error: The current node list is empty. com.branch_international.branch.branch_demo_android

scraper error: The current node list is empty. com.fiksu.fma.android

scraper error: The current node list is empty. com.tinizine.azoomee

scraper error: The current node list is empty. com.tokensapp

scraper error: The current node list is empty. de.merkur24.universalapp

scraper error: Requested resource not found com.pingu.cleansup

scraper error: The current node list is empty. com.mobivate.colourgo

scraper error: The current node list is empty. com.epicactiononline.ffxv.ane

scraper error: The current node list is empty. com.recarga.recarga

jkmbo commented 6 years ago

I guess this has somethign to do with:

$info['title'] = $crawler->filter('[itemprop="name"] > div')->text();

should be $info['title'] = $crawler->filter('[itemprop="name"] > span')->text(); or better yet: $info['title'] = $crawler->filter('[itemprop="name"]')->children()->first()->text();