JoMingyu / google-play-scraper

Google play scraper for Python inspired by <facundoolano/google-play-scraper>
MIT License
757 stars 206 forks source link

[BUG] app() doesn't return any app data , only return review text #135

Closed NupuraW closed 2 years ago

NupuraW commented 2 years ago

google_play_scraper.1.1.0

Describe the bug app_meta = app( 'com.webserveis.batteryinfo', lang='en', # defaults to 'en' country='us' # defaults to 'us' ) doesn't return any data about app. all the fields are None or null

Code Copy and paste the code that have issue.

Expected behavior should return data such as title, developer ,price, description,rating ,intstalls

Additional context Add any other context about the problem here.

kluhan commented 2 years ago

@NupuraW, are you sure you are using the latest version of Google Play Scraper? I just tested it for 10k apps and everything works fine for me. Maybe Google is rolling out an update that is causing the problems, but then I can't fix it until I have the problem too.

NupuraW commented 2 years ago

I'm using the latest version. 1.1.0.

NupuraW commented 2 years ago

does it depend on anything else to pull data? can i use proxy for getting data for an app?

kluhan commented 2 years ago

no, as far as i know there are no dependencies except python >= 3.6.

no, currently there is no way to use a proxy, but you can open a feature request, maybe more people need such a feature. a workaround would be to simply redirect all tcp traffic to a proxy. you could try a tool like redsocks https://github.com/darkk/redsocks if you have a suitable proxy server. another option would be to try a public VPN provider. please let me know if you are successful with this or if you find another solution.

crowley285 commented 2 years ago

I'm having simmilar issue (everything is none) even on the latest version 1.2.0 This only happens when crawling apps for australia (others like us, slovakia, brazil are okay) for example id: "com.arzfoodservice" country "au" https://play.google.com/store/apps/details?id=com.arzfoodservice&hl=en&gl=au

I am crawling from slovakia

rossoft commented 2 years ago

@crowley285 tried right now "com.arzfoodservice" country "au" with 1.2.0 and works OK for me

crowley285 commented 2 years ago

@rossoft Thanks for verifying Interesting, do you think that google doesn't send the same data when crawling from different countries? Like in my case for australia when crawling from slovakia.

rossoft commented 2 years ago

@crowley285 no idea, do you see the page from same laptop via browser? just to check it is not a network problem

crowley285 commented 2 years ago

The browser on the same computer opens the page without problems The scraper also downloads the page but is unable to parse anything from it I'm attaching what is returned by: /google_play_scraper/features/app.py:15 (dom = get(url)) dom.txt

ma9h9di commented 2 years ago

I'm having similar issue (everything is none) even on the latest version 1.2.2 and I solved it by overwrite Detail in ElementSpecs change all ElementSpec(4, ...) by ElementSpec(5, ...) because for me app date in ds:5 not in ds:4

I think Google changes the content of the pages according to the IP address of the users My IP address is inside Iran

rossoft commented 2 years ago

@ma9h9di please check you are really using 1.2.2, as this change ElementSpec(4) to ElementSpec(5) is precisely what is included in 1.2.2

crowley285 commented 2 years ago

I pulled the latest changes and the issue seems to be resolved. So far i did not encounter any issues.

JoMingyu commented 2 years ago

@kluhan & @rossoft huge thanks to you help resolve this issue.