search

crdroidandroid / issue_tracker

all crDroid related issues should go here
11 stars 1 forks source link

bug: SystemUI Memory leak #478

Open Zachhambo opened 3 months ago

Zachhambo commented 3 months ago

Issue type

Other

Device

Poco F2 Pro (lmi)

crDroid version

crDroid 10

Exact version / Build date

10.6/ 2024-07-26

Bug description

While using device, systemUI experiences a memory leak that eventually fills the entire system ram after several hours of usage, resulting in all background (launcher, live wallpaper, music player, matlog) and sometimes foreground apps being killed until systemUI or system is restarted which fixes the RAM hog issue until it fills up again.

Steps to reproduce

Step 1. Observe normal system RAM usage in running services Step 2. Use Phone normally with background apps (eg live wallpaper) Step 3. Wait until background or foreground apps crash Step 4. Observe now bloated system RAM usage Step 5. Restart system UI or entire device to temporarily fix

Relevant log (logcat/build log)

08-03 13:54:33.140 D/ConnectivityService( 1791): releasing NetworkRequest [ REQUEST id=9831, [ Capabilities: INTERNET&NOT_RESTRICTED&TRUSTED&NOT_VCN_MANAGED Uid: 10395 RequestorUid: 10395 RequestorPkg: com.google.android.gms UnderlyingNetworks: Null] ] (release request)
08-03 13:54:33.141 I/ActivityManager( 1791): Process com.google.android.gms.unstable (pid 12191) has died: svc SVC 
08-03 13:54:33.141 I/Zygote  (  955): Process 12191 exited due to signal 9 (Killed)
08-03 13:54:33.141 I/libprocessgroup( 1791): Removed cgroup /sys/fs/cgroup/uid_10395/pid_12191
08-03 13:54:33.141 W/ActivityManager( 1791): Scheduling restart of crashed service com.google.android.gms/.droidguard.DroidGuardService in 28947ms for start-requested
08-03 13:54:33.142 I/WearBitmaps(11839): Returning circular bitmap
08-03 13:54:33.142 I/WatchFacesModel(11839): Got circular bitmap for 16
08-03 13:54:33.142 I/WatchFacesModel(11839): Getting bitmap for 17
08-03 13:54:33.142 I/WatchFacesCommunicator(11839): In getBitmapFromDataItemAsset
08-03 13:54:33.144 W/ActivityManager( 1791): Rescheduling restart of crashed service com.afkettler.pro.earth/com.afkettler.earth.Wallpaper in 6833ms for mem-pressure-event
08-03 13:54:33.144 W/ActivityManager( 1791): Rescheduling restart of crashed service com.google.android.inputmethod.latin/com.android.inputmethod.latin.LatinIME in 26833ms for mem-pressure-event
08-03 13:54:33.144 W/ActivityManager( 1791): Rescheduling restart of crashed service com.google.android.gms/.droidguard.DroidGuardService in 46833ms for mem-pressure-event

Screenshots or videos

Screenshot_20230726-085015_Settings Screenshot_20230712-094723_Settings

Solution

Temporary fix is to restart device or systemui, which reduces system ram usage by 1-2gb and stops apps crashing.

Additional context

Bug has been present since at least 31 May build (likely before). I believe it is present on other devices (hotdog).

Acknowledgements

Hellohi3654 commented 3 months ago

Can confirm I'm running crDroid 10 on my OnePlus 7T Pro And have the same issue Started happening with crDroid 10.5 QPR2 Takes about a day for systemui to use this much memory

Screenshot_20240807-182617_3C Task Manager

After restarting systemui

Screenshot_20240807-182708_3C Task Manager

I'm using kernelsu and have nikgapps full installed

polq1200 commented 3 months ago

I have the same thing on realme gt masterScreenshot_20240730-124950_Termux.png

gwolf2u commented 3 months ago

As 10.6 is now old code since we rebased 10.7, please try latest update 10.7 to see if same issue

Zachhambo commented 3 months ago

10.7 seems to have fixed it, https://github.com/crdroidandroid/android_frameworks_base/commit/891fc4c3b8fec467f92428ce9b35a1e821d10c7d presumably this commit? Thanks Devs!

raystef66 commented 3 months ago

Seems some significant improvement now on LMI. Some users reported to be fixed indeed. Closing this issue. Feel free to re-open otherwise.

gstjee commented 3 months ago

Feel free to re-open otherwise.

can you re-open this issue? Memory leak still a issue on latest synched few hours back build 10.7 . it going to 1.5GB and more. Screenshot_20240813-194624_Settings

Zachhambo commented 3 months ago

Still present on lmi, just took longer to crash than before, Screenshot_20240814-141324_3C Task Manager

raystef66 commented 3 months ago

@Zachhambo Tbh your systemui numbers are quite normal. Issue is when the numbers are in the GB range. At least that's what I've seen on other devices.

gstjee commented 3 months ago

@raystef66 here it is going beyond 2 GB if not killed phone will crash.

Screenshot_20240814-153136_Settings

Screenshot_20240814-153150_Settings

Hellohi3654 commented 2 months ago

Screenshot_20240817-133214_3C Task Manager

Yeah latest 10.7 update for my Oneplus 7T Pro hasn't helped at all Still bad memory leak in systemui

gstjee commented 2 months ago

here we go reached new milestone 2GB xD on today's latest build of 10.7.

Screenshot_20240820-004457_Settings

Hellohi3654 commented 2 months ago

Screenshot_20240822-022951_3C Task Manager

Still got memory leak with latest 10.7 build for my OnePlus 7T Pro Only 13 hours of uptime and already over 1 GB

Hellohi3654 commented 2 months ago

Screenshot_20240823-130112_3C Task Manager

Now over 2gb with 47 hours of uptime Nothing stays running in the background and whole UI is laggy

Zachhambo commented 2 months ago

Screenshot_20240823-164828_3C Task Manager Not sure if this is related but system server has high CPU usage and gets very hot when start Google maps navigation which causes it to continuously crash due to low ram or some other issue. I think I'm done with this rom. Edit: Here is log of google maps dying

2024-08-23-17-43-10.txt

Hellohi3654 commented 2 months ago

I have had this exact same bug in miatoll as well for a while, and I'm glad it's finally been reported.

Btw in 10.7, it has slightly improved.i.e it takes longer for memory to fill up.

But after 70-80h of uptime it gets very difficult to use the device without rebooting

Only takes 2 days for ram usage to be at almost 2gb for systemui for me

Hellohi3654 commented 2 months ago

Does not everyone have this issue or something To me this should be top priority to fix

trivi commented 2 months ago

Please, stop spamming this thread. The developers are definitely aware of the problem, if you have no relevant logs or solutions to provide just up vote the first message. It's not gonna be solved sooner for complaining about it again and again.

ghost commented 2 months ago

Yes this also happens with me most rom affect with that issu now im in 10.7 issu not solv

gwolf2u commented 2 months ago

guys please stop commenting "happening to me too" and stop posting screenshots that are not relevant as you can see I've marked old comments for that purpose it doesn't help debug the issue most of the team members can't reproduce the issue, but since this seems to indeed happen, issue is still open wait for a fix or help debug properly with logs or code if possible all non helping comments will from now be deleted have patience please

gstjee commented 2 months ago

memory info and its map as well for systemui. meminfosysui.log systemui_ramMap.log

Hellohi3654 commented 2 months ago

Got a memory map of systemui

First one is before memory leak Systemui is using 120MB of ram uimaps.txt

This second one is once systemui memory usage has started to grow Systemui is using 360mb of ram uimemleak.txt

I will edit and include another map once systemui usage has reached around 1gb of usage

Edit: I am now running the latest 20240827 test build for my OP 7T Pro Last two tests were on 20240826 Systemui mem usage is 1.76gb sysuimap.txt

raystef66 commented 2 months ago

Check new build on LMI if improved.

PptO07 commented 2 months ago

@gwolf2u @raystef66 @neobuddy89

There is a method to reproduce it just change color pallets multiple time from wall paper picker like keep clicking on various color pallets at same time, spam it hard and then check ram usage from recents it will be less than 1 gb free ram available

Maybe this is only happening on 4.14.x kernels, is what i observed till now

raystef66 commented 2 months ago

@PptO07 LMI is 4.19.x

gstjee commented 2 months ago

There is a method to reproduce it just change color pallets multiple time from wall paper picker

yeah i just tested it and apart from slow rise ram usage due to other leakage, in this way it is easily reproducible . in seconds systemui ram usage shoots 4GB and above.

Maybe this is only happening on 4.14.x kernels

it is happening on many others as well mine kernel is 5.4.x

Hellohi3654 commented 2 months ago

Must still increase without changing colours slowly over time, but yes can confirm on OP 7T Pro changing colours does massively increase ram usage of systemui. Have also noticed lately that Android System:system_server mem usage is up at 400MB does not go down with systemui restart which fixes systemui's mem usage.

neobuddy89 commented 2 months ago

Hi all,

The source has been cleaned up. Please try newer builds built on/after 2nd September and report back.

https://github.com/crdroidandroid/android_frameworks_base/commits/14.0/

Hellohi3654 commented 2 months ago

Framework cleanup has reduced baseline systemui memory usage to 100mb but still mem leak. Changing colours in wallpaper and style still causes it to go up.

Hellohi3654 commented 2 months ago

Theme engine rebase has again reduced baseline systemui memory usage to 70MB but switching colours in wallpaper and style still causes mem usage to go up alot.

gstjee commented 2 months ago

Please try newer builds built on/after 2nd September and report back.

tried few hours back build and still same memory leak issue except smoothness which has been improved a bit.

IMG_20240903_173152_141

asizon commented 2 months ago

Screenshot_20240903-142958_Ajustes All is working fine in my device,miatoll joyeuse

Hellohi3654 commented 2 months ago

On build 20240903 for OnePlus 7T Pro still have memory leak Systemui mem usage is 675MB Memory map of systemui systemui5092024.txt

Memory usage still goes up above 1GB and would get to 2GB eventually if systemui is not restarted but phone would be unusable

gstjee commented 2 months ago

after 2 days of usage without reboot or restarting systemui:

IMG_20240905_093615_322 IMG_20240905_093628_685

neobuddy89 commented 2 months ago

SystemUI usage doesnt shoot up for me on new builds even after changing themes. @gstjee which build date rom you using?

gstjee commented 2 months ago

which build date rom you using?

3 sept evening. let me know if you want one to one test/logs/dumps as device is rooted so it won't be a problem.

gwolf2u commented 2 months ago

which build date rom you using?

3 sept evening. let me know if you want one to one test/logs/dumps as device is rooted so it won't be a problem.

* to highlight once again: as i have shared RAM usage earlier in below link. it seems like native  code being cached but then later on no garbage collection happen for non referenced object (atleast that i can see at frontend usage logs, but i don't have much idea about inside code execution ;p )
  [bug: SystemUI Memory leak #478 (comment)](https://github.com/crdroidandroid/issue_tracker/issues/478#issuecomment-2311719085)

Do you use any mods?

gstjee commented 2 months ago

Do you use any mods?

no mods.

Hellohi3654 commented 2 months ago

I'm on 20240904 for OnePlus 7T Pro Changing themes causes memory to increase but here I haven't touched the theme setting and just let systemui mem usage go up on its own 3C task manager reports it at 2gb of usage Mem map of systemui: sysuimap20240807.txt

raystef66 commented 2 months ago

Latest build I made for LMI (06.09) seems to be doing well. SystemUI not going above 500MB even with tampering theme color under WP. So far so well...

@Zachhambo can you give latest build a try?

Screenshot_20240906-195126_Settings

gstjee commented 2 months ago

@neobuddy89 @gwolf2u @raystef66 After closely observing for more than 1.5 day on two different devices including 1+ 6T, i think found the issue, which is leading to daily slow rise in SystemUI memory leak (except leaking due to changing theme color issue which may or may not be linked to below).

UDFPS's Custom icon and Animations are the cause. In that observation i also found new bug related to UDFPS, will explain one by one:

Issue 1:

Issue 2(seems independent for now but underlying code might be related):

Issue 3:

gwolf2u commented 2 months ago

let's keep one bug at a time do please check for SystemUI memory issue on released 10.8

Hellohi3654 commented 2 months ago

Memory usage still goes up over time for systemui and also still goes up when switching colours themes in wallpaper and style on crdroid 10.8. I'm on build 20240909 for OP 7T Pro

I installed update earlier and sysui mem usage went up to 1.13GB I then switched colour theme a few times in a row and it went to 1.61GB usage.

Here is memory map of sysui at 1.61GB usage: sysui20240911.txt

I have however noticed the memory usage of all other processes is lower then is was before and the zygote bug is gone as well. What I mean by this is processes that used say 120MB before now are only using 80MB according to 3C task manager.

oscar-tm89 commented 2 months ago

Updated to the latest version 10.8, the memory problems persist, even reaching the point of completely saturating the system again and seeing that consumption shot up to 4GB for "System UI". Build 20240909 for OP 7T Pro.

Simply stopping the System UI process eliminates this RAM saturation problem, the process restarts between 200-600MB, a consumption considered normal for this process.

I don't know if the specific configuration of the device will have something to do with it, like some module enabled specifically in Magisk that causes a conflict.

I haven't tested it yet but I don't know if the Dolby Atmos Moto Core module has anything to do with it, especially the one that modifies the system framework ("Moto Core"). I have read that more than one person who has this problem has had this type of modules installed, it may be interfering with the behavior of the system and RAM management. I will do a test and remove the Moto Core module temporarily, if it doesn't work I will remove the Magisk Dolby Atmos Moto g52 and I will review how the system behaves after this.

We would need to confirm if those experiencing this problem have this Magisk module installed.

Hellohi3654 commented 2 months ago

Updated to the latest version 10.8, the memory problems persist, even reaching the point of completely saturating the system again and seeing that consumption shot up to 4GB for "System UI". Build 20240909 for OP 7T Pro.

Simply stopping the System UI process eliminates this RAM saturation problem, the process restarts between 200-600MB, a consumption considered normal for this process.

I don't know if the specific configuration of the device will have something to do with it, like some module enabled specifically in Magisk that causes a conflict.

I haven't tested it yet but I don't know if the Dolby Atmos Moto Core module has anything to do with it, especially the one that modifies the system framework ("Moto Core"). I have read that more than one person who has this problem has had this type of modules installed, it may be interfering with the behavior of the system and RAM management. I will do a test and remove the Moto Core module temporarily, if it doesn't work I will remove the Magisk Dolby Atmos Moto g52 and I will review how the system behaves after this.

We would need to confirm if those experiencing this problem have this Magisk module installed.

I'm on the same device and don't have Moto Core module I use kernelsu and still have the memory leak I have also updated to 20240911 and memory leak is still the same nothing has changed

Edit here is a memory map of sysui at 1.05GB of usage according to 3C task manager: sysui20240912.txt

oscar-tm89 commented 2 months ago

Do you have any dolby module that modifies the system sound or the system framework? I have also just found that the problem persists when removing the "Moto Core" module and after a while, even when playing videos, the system becomes overloaded. In my case, this has happened to me a lot more when watching videos in any browser. I thought it was due to excessive Chrome consumption, so I switched to Firefox and the problem persists. As I mentioned before, the UI System consumes more than 4GB of RAM. The process itself is accumulating cache memory, perhaps from the content that is being played, and it is not deleting it.

I think we should try to find a common configuration pattern for all of us who experience this problem to determine the cause of it. Installed Magisk modules, LSPosed modules, CrDroid specific configuration...

The Magisk modules I have installed in KernelSU are: Dolby Atmos Moto G52, OP7 Pro Camera Enhance (Savitar), Systemless hosts kernelSU module, Zygisk - LSPosed, Zygisk - Detach, Zygisk - Sui, Zygisk Next. LSPosed: 3C Task, Bootloader Spoofer, E-Gobermment Liberator, Enable Screenshot (Disable flag secure), Wa Enhancer, XPL - EX (XPrivacyLua).

Hellohi3654 commented 1 month ago

Do you have any dolby module that modifies the system sound or the system framework? I have also just found that the problem persists when removing the "Moto Core" module and after a while, even when playing videos, the system becomes overloaded. In my case, this has happened to me a lot more when watching videos in any browser. I thought it was due to excessive Chrome consumption, so I switched to Firefox and the problem persists. As I mentioned before, the UI System consumes more than 4GB of RAM. The process itself is accumulating cache memory, perhaps from the content that is being played, and it is not deleting it.

I think we should try to find a common configuration pattern for all of us who experience this problem to determine the cause of it. Installed Magisk modules, LSPosed modules, CrDroid specific configuration...

The Magisk modules I have installed in KernelSU are: Dolby Atmos Moto G52, OP7 Pro Camera Enhance (Savitar), Systemless hosts kernelSU module, Zygisk - LSPosed, Zygisk - Detach, Zygisk - Sui, Zygisk Next. LSPosed: 3C Task, Bootloader Spoofer, E-Gobermment Liberator, Enable Screenshot (Disable flag secure), Wa Enhancer, XPL - EX (XPrivacyLua).

Should probably add nikgapps config if you have modified it or just say crdroid release if you haven't modified it.

I use kernelsu and have: Busybox for Android NDK, Global Optimized GPS file replacer, Play integrity fix, Tricky store, Viper4androidFX repackagedhoch87, Zygisk Next

And my nikgapps config nikgapps.txt

w568w commented 1 month ago

I can confirm that the issue persists in the latest update.

After some research, I found that Android's built-in profiler, Perfetto, is capable of profiling native memory usage of SystemUI and obtaining backtraces, only when using a userdebug build. Additionally, other methods for debugging memory leaks are documented here.

Any ideas? I can help test the memory leak in a userdebug build, but I have no idea how to compile the ROM.

My device is lmi, on 2024/9/14 build (crDroid 10.8), and the memory usage of SystemUI reaches \~2.2GB after several hours (normally 0.6\~0.7GB).

Output of pmap -x <PID of SystemUI>: map.txt

ChocolateLoverRaj commented 1 month ago

This is a "me too" but I think it is useful information. I think I have experienced problems caused by a SystemUI memory leak on my Pixel 2 XL (taimen) in LineageOS too. When I had the march update of LineageOS I did not have any problems. When I got the July update, I started having problems. Since crDroid is based on LineageOS maybe it is a bug that exists for both of them.

When I was using LineageOS with GAPPS, I did not root the phone or install any apps that made customizations. Around once every 2 days I would get the message "System UI is not responding", then the screen would turn off (but the phone was still responsive, and sqeeze for flashlight was working). Then it seemed like the phone restarted but if I checked the up time it was not reset.

Edit: I just experienced the SAME problem on crDroid as I did on LineageOS. Here is what happened:

Here is my issue on LineageOS: https://gitlab.com/LineageOS/issues/android/-/issues/7444. And https://gitlab.com/LineageOS/issues/android/-/issues/7383.

Format data and try again, I don't see these, nor do many other users

So idk why this is happening to my specific phone. Maybe it's a different problem, like a hardware problem?

Hellohi3654 commented 1 month ago

This is a "me too" but I think it is useful information. I think I have experienced problems caused by a SystemUI memory leak on my Pixel 2 XL (taimen) in LineageOS too. When I had the march update of LineageOS I did not have any problems. When I got the July update, I started having problems. Since crDroid is based on LineageOS maybe it is a bug that exists for both of them.

When I was using LineageOS with GAPPS, I did not root the phone or install any apps that made customizations. Around once every 2 days I would get the message "System UI is not responding", then the screen would turn off (but the phone was still responsive, and sqeeze for flashlight was working). Then it seemed like the phone restarted but if I checked the up time it was not reset.

Edit: I just experienced the SAME problem on crDroid as I did on LineageOS. Here is what happened:

  • It was 10:30, but the ambient display showed 9:59
  • Pressing the power button did not show the lock screen
  • I rotated the phone and the ambient display rotated (doesn't normally rotate)
  • Squeeze to toggle flashlight worked
  • I kept pressing the power button
  • The screen went black
  • ~10 seconds later, the crDroid startup animation played
  • It made me enter the pin saying "PIN required after device restarts"
  • I unlocked, went to settings, and up time was 10hr, which means that the phone did not restart. The blue charging light indicator on the phone also never turned off, more evidence that the phone didn't restart So unfortunately I have the same issue on crDroid as I did with LineageOS. I am not sure if it is related to the memory leak.

Here is my issue on LineageOS: https://gitlab.com/LineageOS/issues/android/-/issues/7444. And https://gitlab.com/LineageOS/issues/android/-/issues/7383.

Format data and try again, I don't see these, nor do many other users

So idk why this is happening to my specific phone. Maybe it's a different problem, like a hardware problem?

Sounds like a different issue Could be related but I have never had anything like that happen

Ever since the framework rebase more apps have been running in the background even though allow background is toggled off for some of them. Seems like apps just do what they want since the rebase. Since memory usage is higher by default due to more apps running in the background could be something to do with your issue.

Have you checked average memory usage for systemui in developer settings -> memory -> apps

ChocolateLoverRaj commented 1 month ago

Have you checked average memory usage for systemui in developer settings -> memory -> apps

When I do that, all I see is "Android OS" with 2.6GB used. (Phone has 4GB total).

Also I would have to continuously log the memory usage to find out because I can't predict when the problem will happen.