facundoolano / google-play-scraper

Node.js scraper to get data from Google Play
MIT License
2.3k stars 627 forks source link

Cannot get review past page 110th #97

Closed rungtruc closed 8 years ago

rungtruc commented 8 years ago

I tried to look at "old" reviews of minecraftpe. The app has more than 1.4 million reviews. However, google-play-scraper cannot pull review after page 110th. According to the instruction, it cannot get pass the 40*111 = 4440th review.

facundoolano commented 8 years ago

The library fetches the reviews through the same API as the Google Play site. Are you able to get to those old reviews from the site? If not you won't be able to do it through the lib either (unless there's some other source to get them).

rungtruc commented 8 years ago

Thank you for your prompt response. I don't know if I can reach that much review from the site because it only show a few reviews at a time in a frame (sometime 2 reviews, sometime 5 reviews depending on the length of the text) and I loss my counting pretty soon before I can reach the 4440th review :(.

facundoolano commented 8 years ago

So why exactly you say you can't get past the 110th page? Bare in mind that reviews are separated by language.

rungtruc commented 8 years ago

I set the page argument in your code to 110, it give me a list of 40 reviews. I changed it to 120, then it produce "There is some error fetching the reviews". Then I kept decreasing the page parameter by one to find out the maximum value of the page value. And it appeared to be 111 ( not 110, I just double checked).

facundoolano commented 8 years ago

I see. Well as I said, remember that the 1.4M number means that many people rated the app, not necessarily left a review (most times they don't), and that the reviews are separated by language, so I'm pretty sure you're seeing all the reviews available for the language you are trying.

rungtruc commented 8 years ago

I set option to newest. and use default language (English, I believe). And the same thing happenned. The farthest date it show is July 7.