privacy-tech-lab / gpc-android

Code and dynamic analysis scripts for GPC on Android
https://privacytechlab.org/
MIT License
4 stars 1 forks source link

Running Mitm/Har network captures #85

Closed n-aggarwal closed 9 months ago

n-aggarwal commented 10 months ago

I am opening this issue to document problems that come along as I am collecting data, and to keep track of progress.

n-aggarwal commented 10 months ago

I am running quite a bit behind schedule.The reason for that is that not all the captures are running properly. In a folder of about 45 apps, I am finding only 35 work. Then I have to manually go through the rest of the 10 to find out what went wrong.

The way I am telling if something went wrong is if the “har” file is in bytes, or the “mitm” file is around 11kb for several captures of the app. This is the usual traffic that is captured without running any app. Sometimes, the case is simply that the app does not transmit data, but most of the times it means the apk was not installed properly.

I have found several main reasons for these apks not being installed correctly:

  1. There is a split apk folder and a normal apk for the same app. My code by default looks for the apk, and usually this leads to problems. This one can be fixed by making split apks the default. Screenshot 2023-08-17 at 9 16 46 PM

  2. Sometimes, inside a split apk folder, there are duplicates of the same config files, say “v8a”, and these cause installation problems (notice how the duplicates have different dates as well). Additionally, as can been seen in the screenshots below, there are some stray files with no extension. I have no idea what role these play. Screenshot 2023-08-18 at 2 49 19 AM Screenshot 2023-08-17 at 9 16 08 PM

  3. The third problem is that sometime the apks are entirely missing even though the set should be full. Ex. “com.catdaddy.nba2kmcom.ea.tsb” in “GAME_SPORTS”.

The above mentioned are the frequently occurring problems.

Apart from these, I once saw an xapk, but that may just be a one off. Nevertheless, we need to take care of these problems ASAP for the captures to run efficiently.

n-aggarwal commented 10 months ago

To save time, I propose an alternative. Instead of me looking into what went wrong for each app that didn't work. I will simply note the apps that failed in the "app_fail.txt" file. Then over the course of the next two weeks @wesley-tan can look at those apks and fix them, re-download them, or whatever else needs to be done. Then I can run those apps that are left.

n-aggarwal commented 10 months ago

In this case @wesley-tan should only spend a few minutes on each of the apks that failed. If he can't figure out the issue, then he can simply move on. I will take care of that. His work would be to fix the majority of the apks which fall under one of the 3 categories above. If he can't find the issue, then I will take a look at it.

SebastianZimmeck commented 10 months ago

OK, let's do that.

SebastianZimmeck commented 10 months ago

Here are the current issues:

@wesley-tan is in the lead on fixing the above. Please update @n-aggarwal and @wesley-tan as needed.

SebastianZimmeck commented 9 months ago

Closing since we have the new issue on a more specific point.