topjohnwu / Magisk

The Magic Mask for Android
GNU General Public License v3.0
47.6k stars 12.09k forks source link

Overwriting file in /product/overlay causes issue with MagiskHide (QuickSwitch module makes Google Play Services go crazy) #2107

Closed alvinhochun closed 4 years ago

alvinhochun commented 4 years ago

I just spent a whole few hours trying to figure out what's wrong with it. Apparently the way QuickSwitch replaces file in /product/overlay is causing some issue with MagiskHide and causes Google Play Service (com.google.android.gms) to get stuck trying to read some APKs in /product/overlay (my guess is Play Protect) and ends up continuously utilizing 100% CPU and keep allocating memory.

This is what the QuickSwitch module looks like when Lawnchair is selected:

sargo:/ # ls -l /data/adb/modules/quickstepswitcher/
total 14
drwxrwxrwx 2 root root 3488 2019-11-26 23:12 logs
-rw-r--r-- 1 root root  161 2019-11-26 22:44 module.prop
lrwxrwxrwx 1 root root   18 2019-11-26 23:12 product -> ./system/product
drwxr-xr-x 6 root root 3488 2019-11-26 22:49 system
-rw-r--r-- 1 root root 2220 2019-11-26 22:44 uninstall.sh
sargo:/ # find /data/adb/modules/quickstepswitcher/
/data/adb/modules/quickstepswitcher/
/data/adb/modules/quickstepswitcher/uninstall.sh
/data/adb/modules/quickstepswitcher/module.prop
/data/adb/modules/quickstepswitcher/logs
/data/adb/modules/quickstepswitcher/logs/quickstepswitcher-service.log
/data/adb/modules/quickstepswitcher/logs/quickstepswitcher.log
/data/adb/modules/quickstepswitcher/logs/quickstepswitcher-formatted.log
/data/adb/modules/quickstepswitcher/logs/find.log
/data/adb/modules/quickstepswitcher/logs/quickstepswitcher-old.log
/data/adb/modules/quickstepswitcher/system
/data/adb/modules/quickstepswitcher/system/app
/data/adb/modules/quickstepswitcher/system/app/EQuickSwitch
/data/adb/modules/quickstepswitcher/system/app/EQuickSwitch/EQuickSwitch.apk
/data/adb/modules/quickstepswitcher/system/product
/data/adb/modules/quickstepswitcher/system/product/overlay
/data/adb/modules/quickstepswitcher/system/product/overlay/QuickstepSwitcherOverlay.apk
/data/adb/modules/quickstepswitcher/system/priv-app
/data/adb/modules/quickstepswitcher/system/priv-app/QuickstepSwitcher-ch.deletescape.lawnchair.ci-9f2
/data/adb/modules/quickstepswitcher/system/priv-app/QuickstepSwitcher-ch.deletescape.lawnchair.ci-9f2/ch.deletescape.lawnchair.ci-9f2.apk
/data/adb/modules/quickstepswitcher/system/etc
/data/adb/modules/quickstepswitcher/system/etc/permissions
/data/adb/modules/quickstepswitcher/system/etc/permissions/privapp-permissions-quickstepswitcher.xml
/data/adb/modules/quickstepswitcher/system/etc/sysconfig
/data/adb/modules/quickstepswitcher/system/etc/sysconfig/quickstepswitcher-hiddenapi-package-whitelist.xml
/data/adb/modules/quickstepswitcher/product

This is an excerpt from logcat:

11-26 23:06:30.073 16284 16284 E gle.android.gm: Failed to open APK '/product/overlay/NavigationBarModeGestural/NavigationBarModeGesturalOverlay.apk' I/O error
11-26 23:06:30.074 16284 16284 W ResourcesManager: failed to add overlay path /product/overlay/NavigationBarModeGestural/NavigationBarModeGesturalOverlay.apk
11-26 23:06:30.089 16284 17444 I ChimeraCfgMgr: Sending com.google.android.chimera.IntentOperation.NEW_MODULE Intents: 2,true,false
11-26 23:06:30.232 16284 16299 E System  : Uncaught exception thrown by finalizer
11-26 23:06:30.232 16284 16299 E System  : java.lang.NullPointerException: Attempt to invoke virtual method 'void android.content.res.StringBlock.close()' on a null object reference
11-26 23:06:30.232 16284 16299 E System  :      at android.content.res.ApkAssets.close(ApkAssets.java:195)
11-26 23:06:30.232 16284 16299 E System  :      at android.content.res.ApkAssets.finalize(ApkAssets.java:185)
11-26 23:06:30.232 16284 16299 E System  :      at java.lang.Daemons$FinalizerDaemon.doFinalize(Daemons.java:289)
11-26 23:06:30.232 16284 16299 E System  :      at java.lang.Daemons$FinalizerDaemon.runInternal(Daemons.java:276)
11-26 23:06:30.232 16284 16299 E System  :      at java.lang.Daemons$Daemon.run(Daemons.java:137)
11-26 23:06:30.232 16284 16299 E System  :      at java.lang.Thread.run(Thread.java:919)
11-26 23:06:30.232 16284 16299 E System  : Uncaught exception thrown by finalizer
11-26 23:06:30.232 16284 16299 E System  : java.lang.NullPointerException: Attempt to invoke virtual method 'void android.content.res.StringBlock.close()' on a null object reference
11-26 23:06:30.232 16284 16299 E System  :      at android.content.res.ApkAssets.close(ApkAssets.java:195)
11-26 23:06:30.232 16284 16299 E System  :      at android.content.res.ApkAssets.finalize(ApkAssets.java:185)
11-26 23:06:30.232 16284 16299 E System  :      at java.lang.Daemons$FinalizerDaemon.doFinalize(Daemons.java:289)
11-26 23:06:30.232 16284 16299 E System  :      at java.lang.Daemons$FinalizerDaemon.runInternal(Daemons.java:276)
11-26 23:06:30.232 16284 16299 E System  :      at java.lang.Daemons$Daemon.run(Daemons.java:137)
11-26 23:06:30.232 16284 16299 E System  :      at java.lang.Thread.run(Thread.java:919)
11-26 23:06:30.233 16284 16299 E System  : Uncaught exception thrown by finalizer
11-26 23:06:30.233 16284 16299 E System  : java.lang.NullPointerException: Attempt to invoke virtual method 'void android.content.res.StringBlock.close()' on a null object reference
11-26 23:06:30.233 16284 16299 E System  :      at android.content.res.ApkAssets.close(ApkAssets.java:195)
11-26 23:06:30.233 16284 16299 E System  :      at android.content.res.ApkAssets.finalize(ApkAssets.java:185)
11-26 23:06:30.233 16284 16299 E System  :      at java.lang.Daemons$FinalizerDaemon.doFinalize(Daemons.java:289)
11-26 23:06:30.233 16284 16299 E System  :      at java.lang.Daemons$FinalizerDaemon.runInternal(Daemons.java:276)
11-26 23:06:30.233 16284 16299 E System  :      at java.lang.Daemons$Daemon.run(Daemons.java:137)
11-26 23:06:30.233 16284 16299 E System  :      at java.lang.Thread.run(Thread.java:919)

From a root shell, I can see that the file exists:

sargo:/ # ls -l /product/overlay/NavigationBarModeGestural/
total 16
-rw-r--r-- 1 root root 12638 2009-01-01 08:00 NavigationBarModeGesturalOverlay.apk

However it is inaccessible in MagiskHide's namespace:

sargo:/ # magiskhide exec ls -l /product/overlay/NavigationBarModeGestural/
hide_policy: handling PID=[12028]
hide_policy: Unmounted (/sbin)
hide_policy: Unmounted (/system/etc/permissions)
hide_policy: Unmounted (/system/etc/sysconfig)
hide_policy: Unmounted (/system/app)
hide_policy: Unmounted (/system/priv-app)
hide_policy: Unmounted (/system/bin)
hide_policy: Unmounted (/product/overlay/QuickstepSwitcherOverlay.apk)
hide_policy: Unmounted (/product/overlay/IconPackRoundedSystemUI)
hide_policy: Unmounted (/product/overlay/NavigationBarModeGesturalWideBack)
hide_policy: Unmounted (/product/overlay/PixelConfigOverlay2017.apk)
hide_policy: Unmounted (/product/overlay/IconShapeSquare)
hide_policy: Unmounted (/product/overlay/NfcNci__auto_generated_rro_product.apk)
hide_policy: Unmounted (/product/overlay/PixelDocumentsUIGoogleOverlay)
hide_policy: Unmounted (/product/overlay/IconPackFilledSystemUI)
hide_policy: Unmounted (/product/overlay/NavigationBarMode2Button)
hide_policy: Unmounted (/product/overlay/IconShapeSquircle)
hide_policy: Unmounted (/product/overlay/TeleService__auto_generated_rro_product.apk)
hide_policy: Unmounted (/product/overlay/IconPackCircularLauncher)
hide_policy: Unmounted (/product/overlay/IconPackCircularThemePicker)
hide_policy: Unmounted (/product/overlay/FontNotoSerifSource)
hide_policy: Unmounted (/product/overlay/IconShapeRoundedRect)
hide_policy: Unmounted (/product/overlay/IconShapeTeardrop)
hide_policy: Unmounted (/product/overlay/SystemUIGoogle__auto_generated_rro_product.apk)
hide_policy: Unmounted (/product/overlay/IconPackCircularSettings)
hide_policy: Unmounted (/product/overlay/AccentColorBlack)
hide_policy: Unmounted (/product/overlay/NavigationBarModeGesturalNarrowBack)
hide_policy: Unmounted (/product/overlay/DisplayCutoutEmulationDouble)
hide_policy: Unmounted (/product/overlay/DisplayCutoutEmulationTall)
hide_policy: Unmounted (/product/overlay/ContactsProvider__auto_generated_rro_product.apk)
hide_policy: Unmounted (/product/overlay/BuiltInPrintService__auto_generated_rro_product.apk)
hide_policy: Unmounted (/product/overlay/SettingsProvider__auto_generated_rro_product.apk)
hide_policy: Unmounted (/product/overlay/IconPackFilledSettings)
hide_policy: Unmounted (/product/overlay/TelephonyProvider__auto_generated_rro_product.apk)
hide_policy: Unmounted (/product/overlay/ChinaMobileFrameworksRes)
hide_policy: Unmounted (/product/overlay/SimAppDialog__auto_generated_rro_product.apk)
hide_policy: Unmounted (/product/overlay/IconPackCircularAndroid)
hide_policy: Unmounted (/product/overlay/IconPackRoundedSettings)
hide_policy: Unmounted (/product/overlay/IconPackFilledLauncher)
hide_policy: Unmounted (/product/overlay/NavigationBarMode3Button)
hide_policy: Unmounted (/product/overlay/AccentColorOcean)
hide_policy: Unmounted (/product/overlay/AccentColorOrchid)
hide_policy: Unmounted (/product/overlay/IconPackRoundedAndroid)
hide_policy: Unmounted (/product/overlay/AccentColorCinnamon)
hide_policy: Unmounted (/product/overlay/AccentColorGreen)
hide_policy: Unmounted (/product/overlay/SafetyRegulatoryInfo__auto_generated_rro_product.apk)
hide_policy: Unmounted (/product/overlay/Telecom__auto_generated_rro_product.apk)
hide_policy: Unmounted (/product/overlay/IconPackFilledAndroid)
hide_policy: Unmounted (/product/overlay/PixelConfigOverlay2019Midyear.apk)
hide_policy: Unmounted (/product/overlay/PixelConfigOverlay2018.apk)
hide_policy: Unmounted (/product/overlay/GoogleConfigOverlay.apk)
hide_policy: Unmounted (/product/overlay/IconPackFilledThemePicker)
hide_policy: Unmounted (/product/overlay/NavigationBarModeGestural)
hide_policy: Unmounted (/product/overlay/AccentColorSpace)
hide_policy: Unmounted (/product/overlay/IconPackRoundedLauncher)
hide_policy: Unmounted (/product/overlay/NavigationBarModeGesturalExtraWideBack)
hide_policy: Unmounted (/product/overlay/PixelConfigOverlayCommon.apk)
hide_policy: Unmounted (/product/overlay/GoogleWebViewOverlay.apk)
hide_policy: Unmounted (/product/overlay/framework-res__auto_generated_rro_product.apk)
hide_policy: Unmounted (/product/overlay/AccentColorPurple)
hide_policy: Unmounted (/product/overlay/IconPackCircularSystemUI)
hide_policy: Unmounted (/product/overlay/GooglePermissionControllerOverlay.apk)
hide_policy: Unmounted (/product/overlay/DisplayCutoutEmulationCorner)
hide_policy: Unmounted (/system/etc/hosts)
total 0

Device: Pixel 3a OS: Android 10 (sargo-qp1a.191105.003) Magisk: Latest Canary 20.2-72edbfc4 (20108) (also happened on stable)

alvinhochun commented 4 years ago

Quick guess: missing TMPFS_MNT(product)? https://github.com/topjohnwu/Magisk/blob/72edbfc4559161df2c3edad846c51b797401177d/native/jni/magiskhide/hide_policy.cpp#L70

Mygod commented 4 years ago

This seems like a reasonable guess.

skittles9823 commented 4 years ago

I just updated to the December security patch so I was able to test this and after checking top I was experiencing 100% CPU usage from gms (however I didn't feel any lagging or anything) @paphonb asked me to disable magisk hide and check afterwards and sure enough cpu usage went back to normal.

So if anyone else can test, just disable magisk hide so we can be sure it's a hide issue with the /product partition.

alvinhochun commented 4 years ago

I just updated to the December security patch so I was able to test this and after checking top I was experiencing 100% CPU usage from gms (however I didn't feel any lagging or anything) @paphonb asked me to disable magisk hide and check afterwards and sure enough cpu usage went back to normal.

I'll assume my fix has the same effect for now. If anyone suspect this fix doesn't really fix the CPU usage issue with QuickSwitch I guess we can discuss it back in skittles9823/QuickSwitch#7.

androidacy-user commented 4 years ago

Not resolved in latest canary even though it's supposed to be Gservices still broken when quickswitch is enabled with exact same logs

skittles9823 commented 4 years ago

Can confirm.

gms hidden still causes cpu usage to max out.

osm0sis commented 4 years ago

@alvinhochun should reopen then.

alvinhochun commented 4 years ago

Not resolved in latest canary even though it's supposed to be Gservices still broken when quickswitch is enabled with exact same logs

Same logs as in Failed to open APK '/product/overlay/NavigationBarModeGestural/****************.apk' I/O error? I do not see these log items on latest canary. Are you able to install/update apps from Play Store? If you can that means it's not the same issue.

gms hidden still causes cpu usage to max out.

I did mention in https://github.com/skittles9823/QuickSwitch/issues/7#issuecomment-559989110 that gms still occasionally comes out to eat CPU cycles even with the fix, and I haven't yet figured out why it happens. Easiest way to check is to execute magiskhide exec echo and if it says that it unmounted /product/overlay properly instead of the subdirectories inside, then chances are that the bug referred to in this issue has been fixed and what you're seeing is due to other things somewhere else.

@alvinhochun should reopen then.

Only @topjohnwu can reopen this issue. But when I created this issue I was very specific on addressing one thing that is MagiskHide's unmounting behaviour regarding /product/overlay. If @topjohnwu thinks this issue should be adapted to include the follow-up on gms's behaviour, then please do reopen this. Otherwise, I think we should continue the discussion in skittles9823/QuickSwitch#7.

androidacy-user commented 4 years ago

@alvinhochun when I said exactly same issue I meant exactly the same. Not able to use Google sign-in, nor install nor update apps

androidacy-user commented 4 years ago

And no it doesn't mention anything about unmounted /product/overlay, and magiskhide exec ls -l /product/overlay still lists the relevant overlays that should be hidden

Temporary workaround is to not hide com.google.android.gms and com.google.android.gms:persistent but this is nonetheless undesired behavior, especially if one wants to use Google pay as I can't use it without hiding both those

alvinhochun commented 4 years ago

@alexa-v2 Can you grab some logs (logcat and Magisk) and attach them here? Also include the output of you running the command magiskhide exec ls -R /product/overlay in a root shell.

androidacy-user commented 4 years ago

So if I'm reading this issue correctly it's actually a regression - not only is the issue still present but now it's not hiding the files correctly

androidacy-user commented 4 years ago

@alexa-v2 Can you grab some logs (logcat and Magisk) and attach them here? Also include the output of you running the command magiskhide exec ls -R /product/overlay in a root shell.

I'd have to reset my quickswitch module - I've modded it to use /vendor/overlay instead but nonetheless I can

androidacy-user commented 4 years ago

It would appear it is actually hiding the quickswitch overlay it just never says it unmounted /product/overlay

Otherwise exactly same issues as before

androidacy-user commented 4 years ago

Logs as requested: https://drive.google.com/file/d/1-tfKJFujKAX4pt7OUN-AbByn5GVRekp1/view?usp=drivesdk

alvinhochun commented 4 years ago

@alexa-v2 Seems to me yours is a different issue (different cause), but shows a similar symptom. The biggest difference at first glance is that your device does not have a separate /product partition, i.e. /product is a symlink to /system/product, and thus MagiskHide unmounts /system/product/overlay instead of /product/overlay. This makes me think that this issue should not have affected you in the first place.

In fact your logcat log doesn't even mention /product/overlay except one line from Magisk where MagiskHide unmounts /system/product/overlay.

I see these logcat entries that resembles the ones I had:

12-06 08:19:02.796 W/ziparchive(3106): Unable to open '/system/app/_android.PitchBlack.A07PitchBlackLimeTranslucentSystemUIOFFAndroid10.Android10.apk': No such file or directory
12-06 08:19:02.796 E/.gms.persisten(3106): Failed to open APK '/system/app/_android.PitchBlack.A07PitchBlackLimeTranslucentSystemUIOFFAndroid10.Android10.apk' I/O error
12-06 08:19:02.796 W/ResourcesManager(3106): failed to add overlay path /system/app/_android.PitchBlack.A07PitchBlackLimeTranslucentSystemUIOFFAndroid10.Android10.apk
12-06 08:19:02.796 W/ziparchive(3106): Unable to open '/system/app/_android.SwiftBlack.Lime.PixelandAOSP.apk': No such file or directory
12-06 08:19:02.796 E/.gms.persisten(3106): Failed to open APK '/system/app/_android.SwiftBlack.Lime.PixelandAOSP.apk' I/O error
12-06 08:19:02.799 W/ResourcesManager(3106): failed to add overlay path /system/app/_android.SwiftBlack.Lime.PixelandAOSP.apk
12-06 08:19:02.800 W/ziparchive(3106): Unable to open '/system/app/_android.FlowdorV2.LimeA200HeaderStatusBarSizePanelExpandRoundnessOvalAGradientTypeRadial.apk': No such file or directory
12-06 08:19:02.800 E/.gms.persisten(3106): Failed to open APK '/system/app/_android.FlowdorV2.LimeA200HeaderStatusBarSizePanelExpandRoundnessOvalAGradientTypeRadial.apk' I/O error
12-06 08:19:02.800 W/ResourcesManager(3106): failed to add overlay path /system/app/_android.FlowdorV2.LimeA200HeaderStatusBarSizePanelExpandRoundnessOvalAGradientTypeRadial.apk
12-06 08:19:02.800 W/ziparchive(3106): Unable to open '/system/app/_android.LivDark.GradientLimeViolet33cc99BlackAmoledBackground.Android10.apk': No such file or directory
12-06 08:19:02.800 E/.gms.persisten(3106): Failed to open APK '/system/app/_android.LivDark.GradientLimeViolet33cc99BlackAmoledBackground.Android10.apk' I/O error
12-06 08:19:02.800 W/ResourcesManager(3106): failed to add overlay path /system/app/_android.LivDark.GradientLimeViolet33cc99BlackAmoledBackground.Android10.apk
12-06 08:19:02.800 W/ziparchive(3106): Unable to open '/system/app/_com.google.android.gms.LivDark.Android10.apk': No such file or directory
12-06 08:19:02.800 E/.gms.persisten(3106): Failed to open APK '/system/app/_com.google.android.gms.LivDark.Android10.apk' I/O error
12-06 08:19:02.800 W/ResourcesManager(3106): failed to add overlay path /system/app/_com.google.android.gms.LivDark.Android10.apk

Note that however these refers to files residing in /system/app and not /product/overlay. Considering the first file /system/app/_android.PitchBlack.A07PitchBlackLimeTranslucentSystemUIOFFAndroid10.Android10.apk for example - I see this entry in the Magisk log which indicates that it is added by the "substratum" module:

12-06 13:16:30.210   761   762 I Magisk  : bind_mount: /system/app/_android.PitchBlack.A07PitchBlackLimeTranslucentSystemUIOFFAndroid10.Android10.apk <- /sbin/.magisk/modules/substratum/system/app/_android.PitchBlack.A07PitchBlackLimeTranslucentSystemUIOFFAndroid10.Android10.apk

So it appears to me that MagiskHide is doing the right thing here by hiding these files from gms, but I don't know why gms or ResourcesManager in a "Magisk-hidden" process is looking for these files at all.

It seems to me you should open a separate issue for your case.

(I'm not well versed on either Magisk or the inner workings of modern Android, so @topjohnwu should have a better understanding of the issue than I do.)

androidacy-user commented 4 years ago

@alvinhochun I have a pixel 2 XL which DOES actually have a product partition afaik

GeneralSergal commented 4 years ago

Looks like it's not only /product/overlay folder is affected I getting same drain by .gms.persistant if modules write something in any //overlay folder /system/overlay and /vendor/overlay is affected too

skittles9823 commented 4 years ago

@thevirusua that's interesting. Some QuickSwitch users have found that moving the overlays to /vendor/overlay in the module directory solve the issue for them 🤔

MAO04 commented 4 years ago

Also having this problem with GMS but that's not the strange thing. When i "unhide" GMS it goes away for that but I also have a Samsung S3 watch with Samsung Pay, so I hide three Samsung apps and two of them also go wild. And my choosen wallpaper reverts to default

I'm on a pixel 3xl Dec patch, latest magisk Canary, & QS terminal b11.

Let me know if logs would be useful as I'd have to re-setup to reproduce.

osm0sis commented 4 years ago

It should be noted that this only happens when someone has added the com.google.android.gms activity to the MagiskHide list, either manually or by using the GPay fix module. The MagiskHide default of only having SafetyNet's com.google.android.gms.unstable on the hide list results in a normally working device.

topjohnwu commented 4 years ago

2359

skittles9823 commented 4 years ago

@topjohnwu I don't believe that's the same issue. It's happens to me on my pixel 3a xl, although I don't get the lag su -c top does show high cpu and ram usage and I don't have any images overlaying partitions.

Also if that was the issue, QuickSwitch and navigation bar overlay modules wouldn't be working on a majority of devices.

androidacy-user commented 4 years ago

Related: #2403