search

d4rken-org / sdmaid-se

SD Maid 2/SE is Android's most thorough cleaning tool.
https://play.google.com/store/apps/details?id=eu.darken.sdmse
GNU General Public License v3.0
2.56k stars 393 forks source link

Running out of memory when scanning large amounts of files #837

Closed d4rken closed 4 months ago

d4rken commented 9 months ago

From https://github.com/d4rken-org/sdmaid/issues/6175

Ulefone/Power_Armor_13/Power_Armor_13:12/SP1A.210812.016/1675935266:user/release-keys

Scanning between 44k to 700k files SD Maid runs out of memory.

sdmaidse.txt

The Shizuku crash is likely a side effect:

11-29 06:58:39.684  8751  8751 F DEBUG   : Cmdline: eu.darken.sdmse:SDMSE:Shizuku
11-29 06:58:39.684  8751  8751 F DEBUG   : pid: 8563, tid: 8634, name: RxCachedThreadS  >>> eu.darken.sdmse:SDMSE:Shizuku <<<
11-29 06:58:39.684  8751  8751 F DEBUG   : uid: 2000
11-29 06:58:39.684  8751  8751 F DEBUG   : signal 6 (SIGABRT), code -1 (SI_QUEUE), fault addr --------
11-29 06:58:39.684  8751  8751 F DEBUG   : Abort message: 'Check failed: tlsPtr_.method_verifier == verifier (tlsPtr_.method_verifier=0x0, verifier=0x7529e419a0) '
11-29 06:58:39.684  8751  8751 F DEBUG   :     x0  0000000000000000  x1  00000000000021ba  x2  0000000000000006  x3  0000007529e41620
11-29 06:58:39.684  8751  8751 F DEBUG   :     x4  000000781362e000  x5  000000781362e000  x6  000000781362e000  x7  000000000003c812
11-29 06:58:39.684  8751  8751 F DEBUG   :     x8  00000000000000f0  x9  00000077f93010f0  x10 ffffff00fffffbdf  x11 0000000000000001
11-29 06:58:39.684  8751  8751 F DEBUG   :     x12 0000fffffffff3ff  x13 000000006566709f  x14 0015ef3c1df92820  x15 00003861e4ec5d34
11-29 06:58:39.684  8751  8751 F DEBUG   :     x16 00000077f93a2050  x17 00000077f937eca0  x18 0000007527d82000  x19 00000000000000ac
11-29 06:58:39.684  8751  8751 F DEBUG   :     x20 0000000000002173  x21 00000000000000b2  x22 00000000000021ba  x23 00000000ffffffff
11-29 06:58:39.684  8751  8751 F DEBUG   :     x24 0000000000000002  x25 0000000000000001  x26 0000000000000000  x27 0000007560616000
11-29 06:58:39.684  8751  8751 F DEBUG   :     x28 b400007586fc3c60  x29 0000007529e416a0
11-29 06:58:39.684  8751  8751 F DEBUG   :     lr  00000077f93301cc  sp  0000007529e41600  pc  00000077f93301fc  pst 0000000000001000
11-29 06:58:39.684  8751  8751 F DEBUG   : backtrace:
11-29 06:58:39.684  8751  8751 F DEBUG   :       #00 pc 00000000000511fc  /apex/com.android.runtime/lib64/bionic/libc.so (abort+180) (BuildId: fbbbe88884fc9af7a2c48b60f067a2c3)

SAFDocFile is trying to list the folder contents and runs out of memory:

11-29 08:05:19.412  8504 12479 E AndroidRuntime: java.lang.OutOfMemoryError: Failed to allocate a 24 byte allocation with 70784 free bytes and 69KB until OOM, target footprint 536870912, growth limit 536870912; giving up on allocation because <1% of heap free after GC.
11-29 08:05:19.412  8504 12479 E AndroidRuntime:    at eu.darken.sdmse.common.files.saf.SAFDocFile.listFiles(Unknown Source:89)
11-29 08:05:19.412  8504 12479 E AndroidRuntime:    at eu.darken.sdmse.common.files.saf.SAFGateway$lookupFiles$2.invokeSuspend(Unknown Source:88)
11-29 08:05:19.412  8504 12479 E AndroidRuntime:    at eu.darken.sdmse.common.files.saf.SAFGateway$lookupFiles$2.invoke(Unknown Source:12)
11-29 08:05:19.412  8504 12479 E AndroidRuntime:    at eu.darken.sdmse.common.files.saf.SAFGateway$runIO$2.invokeSuspend(Unknown Source:32)
11-29 08:05:19.412  8504 12479 E AndroidRuntime:    at eu.darken.sdmse.common.files.saf.SAFGateway$runIO$2.invoke(Unknown Source:12)
11-29 08:05:19.412  8504 12479 E AndroidRuntime:    at okio.Okio.startUndispatchedOrReturn(Unknown Source:4)
11-29 08:05:19.412  8504 12479 E AndroidRuntime:    at okio.Okio.withContext(Unknown Source:74)
11-29 08:05:19.412  8504 12479 E AndroidRuntime:    at eu.darken.sdmse.common.files.saf.SAFGateway.runIO(Unknown Source:13)
11-29 08:05:19.412  8504 12479 E AndroidRuntime:    at eu.darken.sdmse.common.files.saf.SAFGateway.lookupFiles(Unknown Source:8)
11-29 08:05:19.412  8504 12479 E AndroidRuntime:    at eu.darken.sdmse.common.files.GatewaySwitch$lookupFiles$3.invokeSuspend(Unknown Source:32)
11-29 08:05:19.412  8504 12479 E AndroidRuntime:    at eu.darken.sdmse.common.files.GatewaySwitch$lookupFiles$3.invoke(Unknown Source:12)
11-29 08:05:19.412  8504 12479 E AndroidRuntime:    at eu.darken.sdmse.common.files.GatewaySwitch.useGateway(Unknown Source:86)
11-29 08:05:19.412  8504 12479 E AndroidRuntime:    at eu.darken.sdmse.common.files.GatewaySwitch.lookupFiles(SourceFile:1)
11-29 08:05:19.412  8504 12479 E AndroidRuntime:    at eu.darken.sdmse.common.files.GatewaySwitch.lookupFiles(SourceFile:2)
11-29 08:05:19.412  8504 12479 E AndroidRuntime:    at eu.darken.sdmse.common.files.PathTreeFlow.collectSafely(Unknown Source:244)
11-29 08:05:19.412  8504 12479 E AndroidRuntime:    at kotlinx.coroutines.flow.AbstractFlow.collect(Unknown Source:68)
11-29 08:05:19.412  8504 12479 E AndroidRuntime:    at coil.util.-Logs.toCollection(Unknown Source:61)
11-29 08:05:19.412  8504 12479 E AndroidRuntime:    at coil.util.-Logs.toList$default(Unknown Source:5)
11-29 08:05:19.412  8504 12479 E AndroidRuntime:    at eu.darken.sdmse.appcleaner.core.scanner.AppScanner.readAppDirs(Unknown Source:482)
11-29 08:05:19.412  8504 12479 E AndroidRuntime:    at eu.darken.sdmse.appcleaner.core.scanner.AppScanner.scan(Unknown Source:1225)
11-29 08:05:19.412  8504 12479 E AndroidRuntime:    at eu.darken.sdmse.appcleaner.core.scanner.AppScanner$scan$1.invokeSuspend(Unknown Source:12)
11-29 08:05:19.412  8504 12479 E AndroidRuntime:    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(Unknown Source:8)
11-29 08:05:19.412  8504 12479 E AndroidRuntime:    at kotlinx.coroutines.DispatchedTask.run(Unknown Source:107)
11-29 08:05:19.412  8504 12479 E AndroidRuntime:    at com.android.billingclient.api.zzl.run(Unknown Source:33)
11-29 08:05:19.412  8504 12479 E AndroidRuntime:    at kotlinx.coroutines.scheduling.TaskImpl.run(Unknown Source:2)
11-29 08:05:19.412  8504 12479 E AndroidRuntime:    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(Unknown Source:93)
11-29 08:05:19.412  8504 12479 E AndroidRuntime:    Suppressed: kotlinx.coroutines.internal.DiagnosticCoroutineContextException: [LazyStandaloneCoroutine{Cancelling}@ffed944, Dispatchers.IO]

when constructing the URIs

11-29 07:57:03.108  2979 12146 E AndroidRuntime: FATAL EXCEPTION: SDM:Thread:1
11-29 07:57:03.108  2979 12146 E AndroidRuntime: Process: eu.thedarken.sdm, PID: 2979
11-29 07:57:03.108  2979 12146 E AndroidRuntime: java.lang.OutOfMemoryError: Failed to allocate a 88 byte allocation with 4029784 free bytes and 3935KB until OOM, target footprint 536870912, growth limit 536870912; giving up on allocation because <1% of heap free after GC.
11-29 07:57:03.108  2979 12146 E AndroidRuntime:    at java.util.Arrays.copyOf(Arrays.java:3578)
11-29 07:57:03.108  2979 12146 E AndroidRuntime:    at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:177)
11-29 07:57:03.108  2979 12146 E AndroidRuntime:    at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:638)
11-29 07:57:03.108  2979 12146 E AndroidRuntime:    at java.lang.StringBuilder.append(StringBuilder.java:226)
11-29 07:57:03.108  2979 12146 E AndroidRuntime:    at android.net.Uri.encode(Uri.java:1922)
11-29 07:57:03.108  2979 12146 E AndroidRuntime:    at android.net.Uri.encode(Uri.java:1864)
11-29 07:57:03.108  2979 12146 E AndroidRuntime:    at android.net.Uri$PathPart.appendDecodedSegment(Uri.java:2255)
11-29 07:57:03.108  2979 12146 E AndroidRuntime:    at android.net.Uri$Builder.appendPath(Uri.java:1475)
11-29 07:57:03.108  2979 12146 E AndroidRuntime:    at android.provider.DocumentsContract.buildDocumentUriUsingTree(DocumentsContract.java:1012)
11-29 07:57:03.108  2979 12146 E AndroidRuntime:    at a1.z.N(Unknown Source:205)
11-29 07:57:03.108  2979 12146 E AndroidRuntime:    at tb.a$b.a(SourceFile:20)
11-29 07:57:03.108  2979 12146 E AndroidRuntime:    at ed.e.getValue(Unknown Source:20)
11-29 07:57:03.108  2979 12146 E AndroidRuntime:    at tb.a.isEmpty(Unknown Source:2)
11-29 07:57:03.108  2979 12146 E AndroidRuntime:    at la.d$a.a(SourceFile:210)
11-29 07:57:03.108  2979 12146 E AndroidRuntime:    at eu.thedarken.sdm.systemcleaner.core.filter.b.match(Unknown Source:10)
11-29 07:57:03.108  2979 12146 E AndroidRuntime:    at h3.j.a(SourceFile:361)
11-29 07:57:03.108  2979 12146 E AndroidRuntime:    at tb.b.a(SourceFile:162)
11-29 07:57:03.108  2979 12146 E AndroidRuntime:    at tb.b.n(SourceFile:70)
11-29 07:57:03.108  2979 12146 E AndroidRuntime:    at ob.a.n(Unknown Source:7)
11-29 07:57:03.108  2979 12146 E AndroidRuntime:    at mb.d0.n(SourceFile:80)
11-29 07:57:03.108  2979 12146 E AndroidRuntime:    at ja.b.M(SourceFile:659)
11-29 07:57:03.108  2979 12146 E AndroidRuntime:    at i8.a.I(SourceFile:5)
11-29 07:57:03.108  2979 12146 E AndroidRuntime:    at ja.b.I(SourceFile:39)
11-29 07:57:03.108  2979 12146 E AndroidRuntime:    at i8.e$b.run(SourceFile:38)
11-29 07:57:03.108  2979 12146 E AndroidRuntime:    at h3.d.run(SourceFile:43)
11-29 07:57:03.108  2979 12146 E AndroidRuntime:    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
11-29 07:57:03.108  2979 12146 E AndroidRuntime:    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:644)
11-29 07:57:03.108  2979 12146 E AndroidRuntime:    at h3.d.run(SourceFile:130)
11-29 07:57:03.108  2979 12146 E AndroidRuntime:    at java.lang.Thread.run(Thread.java:1012)
d4rken commented 4 months ago

1127 will improve upon this. Please re-open if this is still an issue after the next update.