facebook / fresco

An Android library for managing images and the memory they use.
https://frescolib.org/
MIT License
17.07k stars 3.75k forks source link

Out of memory during trimming memory #2468

Open GlebMaltsev opened 4 years ago

GlebMaltsev commented 4 years ago

Description

We receiving OOM crashes from production app. Their appear while we're trying to trim memory. Our implementation are the same as described here: https://github.com/facebook/fresco/issues/2136#issuecomment-397608264

Stack trace:

Fatal Exception: java.lang.OutOfMemoryError: Failed to allocate a 280364296 byte allocation with 8388608 free bytes and 197MB until OOM, max allowed footprint 204217504, growth limit 402653184
       at java.util.Arrays.copyOf(Arrays.java:3139)
       at java.util.Arrays.copyOf(Arrays.java:3109)
       at java.util.ArrayList.grow(ArrayList.java:275)
       at java.util.ArrayList.ensureExplicitCapacity(ArrayList.java:249)
       at java.util.ArrayList.ensureCapacityInternal(ArrayList.java:241)
       at java.util.ArrayList.add(ArrayList.java:467)
       at com.facebook.imagepipeline.cache.CountingMemoryCache.trimExclusivelyOwnedEntries(CountingMemoryCache.java:411)
       at com.facebook.imagepipeline.cache.CountingMemoryCache.trim(CountingMemoryCache.java:349)
       at com.myapp.utils.FrescoMemoryTrimmableRegistry.trim(FrescoMemoryTrimmableRegistry.java:23)
       at com.myapp.MyApp.onTrimMemory(MyApp.java:200)
       at android.app.ActivityThread.handleTrimMemory(ActivityThread.java:5440)
       at ...

MyApp.java is our Application class with override of onTrimMemory() method in the same way as in comment I posted above. FrescoMemoryTrimmableRegistry.java:

class FrescoMemoryTrimmableRegistry : MemoryTrimmableRegistry {

    private val trimmables = LinkedList<MemoryTrimmable>()

    override fun registerMemoryTrimmable(trimmable: MemoryTrimmable) {
        trimmables.add(trimmable)
    }

    override fun unregisterMemoryTrimmable(trimmable: MemoryTrimmable) {
        trimmables.remove(trimmable)
    }

    @Synchronized
    fun trim(trimType: MemoryTrimType) {
        for (trimmable in trimmables) {
            trimmable.trim(trimType)
        }
    }
}

So crash appear sometimes when app in background (46%) and sometimes when in foreground (54%). According to the message of the crash, system need an insane amount of memory to.. trim itself. For example:

Samsung Galaxy J7 Prime (3 GB RAM total): Fatal Exception: java.lang.OutOfMemoryError: Failed to allocate a 420546432 byte allocation with 8388608 free bytes and 231MB until OOM, max allowed footprint 302209776, growth limit 536870912

Huawei Y7 Prime (2019) (3 GB RAM total): Failed to allocate a 420546432 byte allocation with 8339456 free bytes and 236MB until OOM, max allowed footprint 296859888, growth limit 536870912

HUAWEI Nova 2 Lite (3 GB RAM total): Failed to allocate a 420546432 byte allocation with 25165824 free bytes and 234MB until OOM, max allowed footprint 316307200, growth limit 536870912

Motorola Moto G (5th gen) (2 GB RAM total): Failed to allocate a 280364296 byte allocation with 8388608 free bytes and 197MB until OOM, max allowed footprint 204217504, growth limit 402653184

Motorola Moto E5 Play (1 GB RAM total): Failed to allocate a 186909536 byte allocation with 25149440 free bytes and 125MB until OOM, max allowed footprint 162280960, growth limit 268435456

One more thing I found that as you can see above number of required memory is the same across devices with the same memory: ~178 MB for 1 GB RAM devices, 267 MB for 2 GB RAM devices and 401 MB for 3+ GB RAM devices. Probably it's coincidence: app attempts to increase the size of the array and due to growth factor it growth with such big steps. And low memory devices give up on step N, 2-GB devices give up on step N+1, and so on.

Reproduction

We're not able to reproduce this bug on our devices. However it production we have 5% of users who faced with this issue.

Able to reproduce in 100% cases. Please check first comment below for more details.

Additional Information

Could be related: issues start to occur in our latest app release. We didn't change anything related to Fresco (like lib version on way of using Fresco API) except adding one more line:

Fresco.getImagePipeline().evictFromMemoryCache(uri)

We call this in very rare cases (for current moment we have only 92 calls of this).

(Not related since I able to reproduce crash without this line)

GlebMaltsev commented 4 years ago

During debugging I was able to reproduce the issue. Issue occur only in cases when I use the following method to download raw bitmap images from the url:

val uri = Uri.parse("url_to_png_image_resource in the WWW")
val imageRequest = ImageRequest.fromUri(uri)

Fresco
       .getImagePipeline()
       .fetchDecodedImage(imageRequest, null)
       .subscribe(object : BaseBitmapDataSubscriber() {
               override fun onNewResultImpl(bitmap: Bitmap?) {
                         // Bitmap size = 87 KB

                         // Send item to create a Texture in current thread
                         onBitmapReady(bitmap) 
                        }
               }

              override fun onFailureImpl(dataSource: DataSource<CloseableReference<CloseableImage>>)
                            = onBitmapReady(null)
              }, 
CallerThreadExecutor.getInstance())

It's enough to make only 1 call of this method for small (> 100KB) bitmap to reproduce the crash. After I downloaded bitmap and successfully display it on the UI, I move app to background and then restore it. After that crash appear (with attempting to allocate 400MB memory during trimming memory)

Please note that adding .setResizeOptions(ResizeOptions(50, 50)) will not give any results (except decreasing bitmap size to 9KB) - after I recover app from foreground, crash appear.

GlebMaltsev commented 4 years ago

After commenting out onBitmapReady(bitmap) call inside onNewResultImpl() callback crash is gone. It seems that the problem is actually in the wrong work with bitmaps in onBitmapReady(bitmap).

After moving from BaseBitmapDataSubscriber to BaseDataSubscriber problem is gone. I was able to rework logic in my app and now I'm using Closeable References for passing bitmaps across the app.

However, I still have a one last question: how it's possible that passing single 100 KB Bitmap out from BaseBitmapDataSubscriber.onNewResultImpl(Bitmap) to the app in 100% cases will cause OOM due to attempt of allocating hundreds of megabytes of memory during trimming memory in Fresco cache when user moves app to background and then recover it?

defHLT commented 4 years ago

Thanks for the report and investigation. This indeed looks strange. Are you sure there are no huge bitmaps getting cached? Could you please check with Flipper plugin (or by debugging CountingMemoryCache#mCachedEntries) that cached bitmaps are of the reasonable size.

GlebMaltsev commented 4 years ago

Here's a result of executing the following code at the moment right before call CountingMemoryCache.trim() which would lead to crash:

Log.e("memory cache amount: " + trimmable.count)
Log.e("memory cache size: " + trimmable.sizeInBytes)
Log.e("memory cache evictionQueueCount: " + trimmable.evictionQueueCount)
Log.e("memory cache evictionQueueSizeInBytes: " + trimmable.evictionQueueSizeInBytes)
memory cache amount: 73
memory cache size: 25025040
memory cache evictionQueueCount: 73
memory cache evictionQueueSizeInBytes: 25025040

And crash message is Failed to allocate a 420546432 byte allocation with 8388608 free bytes and 232MB until OOM, target footprint 300947288, growth limit 536870912

I also debugged CountingMemoryCache#mCachedEntries and I wasn't find any suspicious bitmaps - regular bitmaps as we use in our app.

GlebMaltsev commented 4 years ago

Just to be more precise, I reproduced crash again and printed out dimens for each bitmap in cache before crash appear.

Show me logs ``` index: 0, width: 240, height: 240 index: 1, width: 200, height: 200 index: 2, width: 150, height: 150 index: 3, width: 150, height: 150 index: 4, width: 150, height: 150 index: 5, width: 300, height: 300 index: 6, width: 150, height: 150 index: 7, width: 250, height: 250 index: 8, width: 250, height: 250 index: 9, width: 250, height: 250 index: 10, width: 250, height: 250 index: 11, width: 250, height: 250 index: 12, width: 250, height: 250 index: 13, width: 348, height: 348 index: 14, width: 128, height: 128 index: 15, width: 128, height: 128 index: 16, width: 150, height: 150 index: 17, width: 128, height: 128 index: 18, width: 128, height: 128 index: 19, width: 150, height: 150 index: 20, width: 150, height: 150 index: 21, width: 128, height: 128 index: 22, width: 150, height: 150 index: 23, width: 240, height: 240 index: 24, width: 150, height: 150 index: 25, width: 388, height: 388 index: 26, width: 576, height: 576 index: 27, width: 240, height: 240 index: 28, width: 128, height: 128 index: 29, width: 144, height: 144 index: 30, width: 150, height: 150 index: 31, width: 315, height: 315 index: 32, width: 527, height: 527 index: 33, width: 128, height: 128 index: 34, width: 150, height: 150 index: 35, width: 540, height: 540 index: 36, width: 150, height: 150 index: 37, width: 100, height: 100 index: 38, width: 128, height: 128 index: 39, width: 150, height: 150 index: 40, width: 150, height: 150 index: 41, width: 150, height: 150 index: 42, width: 150, height: 150 index: 43, width: 150, height: 150 index: 44, width: 128, height: 128 index: 45, width: 228, height: 228 index: 46, width: 150, height: 150 index: 47, width: 150, height: 150 index: 48, width: 540, height: 540 index: 49, width: 335, height: 335 index: 50, width: 150, height: 150 index: 51, width: 150, height: 150 index: 52, width: 352, height: 352 index: 53, width: 150, height: 150 index: 54, width: 150, height: 150 index: 55, width: 512, height: 512 index: 56, width: 340, height: 340 index: 57, width: 128, height: 128 index: 58, width: 128, height: 128 index: 59, width: 150, height: 150 index: 60, width: 240, height: 240 index: 61, width: 150, height: 150 index: 62, width: 128, height: 128 index: 63, width: 150, height: 150 index: 64, width: 360, height: 360 index: 65, width: 540, height: 540 index: 66, width: 451, height: 451 index: 67, width: 363, height: 363 index: 68, width: 348, height: 348 index: 69, width: 452, height: 452 index: 70, width: 150, height: 150 index: 71, width: 128, height: 128 index: 72, width: 500, height: 500 index: 73, width: 128, height: 128 index: 74, width: 350, height: 250 index: 75, width: 150, height: 150 index: 76, width: 150, height: 150 index: 77, width: 150, height: 150 index: 78, width: 361, height: 361 index: 79, width: 150, height: 150 index: 80, width: 540, height: 540 index: 81, width: 128, height: 128 index: 82, width: 240, height: 240 index: 83, width: 150, height: 150 index: 84, width: 240, height: 240 index: 85, width: 337, height: 337 index: 86, width: 588, height: 588 index: 87, width: 360, height: 360 index: 88, width: 599, height: 599 index: 89, width: 128, height: 128 index: 90, width: 396, height: 396 index: 91, width: 128, height: 128 index: 92, width: 500, height: 500 index: 93, width: 150, height: 150 index: 94, width: 240, height: 240 index: 95, width: 240, height: 240 index: 96, width: 128, height: 128 index: 97, width: 150, height: 150 index: 98, width: 150, height: 150 index: 99, width: 240, height: 240 index: 100, width: 486, height: 486 index: 101, width: 128, height: 128 index: 102, width: 150, height: 150 index: 103, width: 135, height: 135 index: 104, width: 150, height: 150 index: 105, width: 240, height: 240 index: 106, width: 150, height: 150 index: 107, width: 240, height: 240 index: 108, width: 430, height: 430 index: 109, width: 218, height: 218 index: 110, width: 450, height: 450 index: 111, width: 404, height: 404 index: 112, width: 150, height: 150 index: 113, width: 588, height: 588 index: 114, width: 150, height: 150 index: 115, width: 396, height: 396 memory cache amount: 116 memory cache size: 37036884 memory cache evictionQueueCount: 116 memory cache evictionQueueSizeInBytes: 37036884 ```
stale[bot] commented 4 years ago

Hey there, it looks like there has been no activity on this issue recently. Has the issue been fixed, or does it still require the community's attention? This issue may be closed if no further activity occurs. You may also label this issue as "bug" or "enhancement" and I will leave it open. Thank you for your contributions.

GlebMaltsev commented 4 years ago

I guess we can mark this as "bug"

zhangzhige commented 3 years ago

I have encounter this bug,how to resolve it? @GlebMaltsev

zhangzhige commented 3 years ago

image image image

this can fix this bug,but need to modify source code @oprisnik @GlebMaltsev

zhangzhige commented 3 years ago

Maybe It cause by a Dead cycle,In a dead cycle,ceaselessly add data to arraylist,It will cause oom by allocation > 100M memory。so In this code image if mExclusiveEntries.getSizeInBytes() > size but mExclusiveEntries is empty,then it will goto dead cycle to add null data to arraylist。 why or when mExclusiveEntries getSizeInBytes has wrong value,I dont know

zhangzhige commented 3 years ago

if CloseableImage getSizeInBytes method,when add to cache getSizeInBytes return real value,when remove to getSizeInBytes return 0,It will cause mExclusiveEntries is empty but mExclusiveEntries.mSizeInBytes is a big value。 image

seecaoxing commented 3 years ago

why don't use setRegisterLruBitmapPoolAsMemoryTrimmable method set lrubitmappool