Incomplete Data

[rehmatworks]

The latest details i.e. s.details('com.whatsapp') returned by the API doesn't contain category information. In addition, the details summary of the apps in search results are also not clean. Is this something changed at Google's end or the recent updates made to gpapi resulted in those issues? Does anyone else also face the same issues?

[IRGC]

Yes category seems removed but it's still available in different protobuf message index number.

[rehmatworks]

@korangu Thanks for your finding. Do you know the index number? Or can you share the updated proto file if you have got this working?

[komeilkamal]

I have same issue. appCategory and appType removed from detail response.

[rehmatworks]

Yep, that's missing. Hopefully @NoMore201 will fix the issue as this is a breaking one.

[IRGC]

message BadgeLinkContainer {
  optional string link = 2;
  optional catInfo catinfo = 7;
}
message catInfo {
  optional catDetails catdetails = 5;  
}
message catDetails {
  optional string category = 4;  
}

@rehmatworks @komeilkamal @NoMore201 may be you could name it better and compile new .proto to generate the googleplay_pb2.py

[SeBsZ]

I'm also looking forward to this. Thanks in advance for your time!

[rehmatworks]

@korangu Thank you for the input but I cannot get this working. I still cannot get category info in DetailsResponse

[IRGC]

@rehmatworks check under Badge in DetailsResponse

[rehmatworks]

@korangu thank you so much. BTW parseProtobufObj raises an exception when I try to parse badge. I feel that we should parse the responses manually in utils.py to ensure that the data produced by gpapi always remains consistent.

[rehmatworks]

@korangu Thank you for your help so far, can you please help me with appType as well? If you get that info for me as well, that'll solve this issue for now.

[IRGC]

@rehmatworks check DetailsResponse->DocV2->DocumentDetails->AppDetails->appType

[rehmatworks]

That contained the info for games only. BTW thank you for your support. I used that as a condition to guess the second type 'Application'. Thank you again.

[rehmatworks]

I'll keep this issue open for @NoMore201 maybe he will consider fixing this issue :)

[SeBsZ]

I'm still wondering if a new version of the protobuf could possible add the old categories back in, @NoMore201? Right now, using the 'Badge' node gives slightly different category names than before. For example, 'FAMILY_ACTION' is now 'Family Friendly', etc.

Your feedback is appreciated. If there is no other way, and the old category is simply gone, then I might need to implement some kind of conversion table.

[NoMore201]

The project uses protobuf and device definitions from play-store-api. Since I don't have much time to dedicate to this project at this time, and I'm not using it actively, I'll try to update protobuf file this weekend, but if anyone is willing to help me with a PR it would be better :)

[SeBsZ]

It would seem that even the 'badge' node has now been completely removed from the Play Store reponse object. At the moment there seems to be no way to get the Play Store category the app belongs to. @NoMore201 have you had time to update the protobuf, do you think that would help? I wouldn't mind updating it and creating a PR myself, but you'd have to provide me with some instructions because I don't know how to do this.

[archon810]

@NoMore201 Any updates please?

[rehmatworks]

Looks like @NoMore201 doesn't have time for this. I did some research and found the original author's post that describes how to recover .proto file from Google Play Store app. If anyone of you know the protocol buffers, then it might be an easy job for them to get the .proto file from the latest Play Store app. Here is the post: https://www.segmentationfault.fr/publications/reversing-google-play-and-micro-protobuf-applications/ @SeBsZ

[SeBsZ]

@rehmatworks thanks. After a long struggle, I managed to get the script to run on an old Play Store APK. This was time-consuming because I needed to use a 6 year old version of Androguard, as later versions were not compatible due to all sorts of changes.

Unfortunately, all work was for nothing. It only worked on a really old Play Store APK (3.7.15). Newer APKs changed the class structure and even later APKs use obfuscation. I see no way to extract a newer protobuf. If anyone has another idea, let me know.

[NoMore201]

IMO the only way to extract newer protobuf definitions is sniff the traffic between a rooted device using google apps and the google servers, which is what I did when I started improving the original project. This process requires:

• A rooted device with google apps where you need to install a custom certificate in order to sniff the traffic (I used mitmproxy )
• Working around some annoying obstacles, like
  • Play Store certificate pinning
  • Installing certificates in /system/etc/security on a rooted device (you need to respect the format of other certificates installed, it requires a bit of work)
• Time

Since unfortunately I don't have any of the three requirements, I am not able to do it alone. On the other hand I will gladly accept some help by other developers willing to improve this library.

Also the current protobuf definition, taken from play-store-api repository, has some field named "unknown", so it makes me think that also the original author used this method to extract those definitions.

[rehmatworks]

@SeBsZ Thank you for sharing the outcome and for your hard work. @NoMore201 I hope community will help address this issues. I'll also spend time on this and will see if I succeed in finding any useful solutions.

[archon810]

Has anyone had time to figure out how to get the latest updates to the protobuf here?