bumptech / glide

An image loading and caching library for Android focused on smooth scrolling
https://bumptech.github.io/glide/
Other
34.63k stars 6.12k forks source link

Out of memory Exception when downloading some large images #825

Closed tom-anders closed 8 years ago

tom-anders commented 8 years ago

So I'm trying to download and save many (~1800) full size images as part of an IntentService. This is the code:

Comic comic = new Comic(i, getApplicationContext());
    String url = comic.getComicData()[2];
    Bitmap mBitmap = Glide.with(this)
        .load(url)
        .asBitmap()
        .into(-1, -1)
        .get();
try {
    File sdCard = prefHelper.getOfflinePath();
    File dir = new File(sdCard.getAbsolutePath() + OFFLINE_PATH);
    dir.mkdirs();
    File file = new File(dir, String.valueOf(i) + ".png");
    FileOutputStream fos = new FileOutputStream(file);
    mBitmap.compress(Bitmap.CompressFormat.PNG, 100, fos);
    fos.flush();
    fos.close();
} catch (Exception e) {
    Log.e("Error", "Saving to external storage failed")}
}

Now, for me this works fine, but a user sent me a log where Glide throws an OutOfMemoryException after about 800 downloaded images:

      19:32:33.051 <9885>[PriorityExecutor]: Request threw uncaught throwable
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError
    at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:223)
    at java.util.concurrent.FutureTask.get(FutureTask.java:82)
    at com.bumptech.glide.load.engine.executor.FifoPriorityThreadPoolExecutor.afterExecute(FifoPriorityThreadPoolExecutor.java:96)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1084)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:569)
    at java.lang.Thread.run(Thread.java:856)
    at com.bumptech.glide.load.engine.executor.FifoPriorityThreadPoolExecutor$DefaultThreadFactory$1.run(FifoPriorityThreadPoolExecutor.java:118)
Caused by: java.lang.OutOfMemoryError
    at android.graphics.BitmapFactory.nativeDecodeStream(Native Method)
    at android.graphics.BitmapFactory.decodeStream(BitmapFactory.java:532)
    at com.bumptech.glide.load.resource.bitmap.Downsampler.decodeStream(Downsampler.java:329)
    at com.bumptech.glide.load.resource.bitmap.Downsampler.downsampleWithSize(Downsampler.java:220)
    at com.bumptech.glide.load.resource.bitmap.Downsampler.decode(Downsampler.java:153)
    at com.bumptech.glide.load.resource.bitmap.StreamBitmapDecoder.decode(StreamBitmapDecoder.java:50)
    at com.bumptech.glide.load.resource.bitmap.StreamBitmapDecoder.decode(StreamBitmapDecoder.java:19)
    at com.bumptech.glide.load.resource.bitmap.ImageVideoBitmapDecoder.decode(ImageVideoBitmapDecoder.java:39)
    at com.bumptech.glide.load.resource.bitmap.ImageVideoBitmapDecoder.decode(ImageVideoBitmapDecoder.java:20)
    at com.bumptech.glide.load.engine.DecodeJob.decodeFromSourceData(DecodeJob.java:190)
    at com.bumptech.glide.load.engine.DecodeJob.decodeSource(DecodeJob.java:177)
    at com.bumptech.glide.load.engine.DecodeJob.decodeFromSource(DecodeJob.java:128)
    at com.bumptech.glide.load.engine.EngineRunnable.decodeFromSource(EngineRunnable.java:122)
    at com.bumptech.glide.load.engine.EngineRunnable.decode(EngineRunnable.java:101)
    at com.bumptech.glide.load.engine.EngineRunnable.run(EngineRunnable.java:58)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:442)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:305)
    at java.util.concurrent.FutureTask.run(FutureTask.java:137)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1076)
    ... 3 more

Sorry, I can't get the stack trace formatted properly for some reason

What am I doing wrong here?

TWiStErRob commented 8 years ago

Is this an IntentService? If so that's probably wrapping ApplicationContext. Which means that each load started will be attached to the app, meaning there's no automatic freeing (can't attach Glide's magic fragment).

If I'm right: you're not freeing Bitmaps, I wonder how it works for you. I think you can help Glide by explicitly letting go:

FutureTarget<Bitmap> target = Glide
        .with(this)
        .load(url)
        .asBitmap()
        .into(-1, -1)
;
try {
    FileOutputStream fos = new FileOutputStream(file);
    target.get().compress(...);
    ...
} catch (Exception e) {
    Log.e("Error", "Saving to external storage failed");
} finally {
    Glide.clear(target);
}

There's also a lesser-known Glide API for this use case:

FutureTarget<byte[]> target = Glide
        .with(this)
        .load(url)
        .asBitmap()
        .toBytes(CompressFormat.PNG, 100) // use Glide to wrap Bitmap.compress
        //.atMost().override(2000, 2000) // useful if you want to save space on device
        // (max(display width/height) is a good target if you don't use a zooming ImageView)
        .format(DecodeFormat.PREFER_ARGB_8888) // make sure you're not losing quality, default is 565
        .diskCacheStrategy(DiskCacheStrategy.NONE) // don't populate the cache if not used
        .skipMemoryCache(true) // there's no point in querying or returning loaded stuff to memory cache
        .into(Target.SIZE_ORIGINAL, Target.SIZE_ORIGINAL) // -1 is not valid
;
try {
    FileOutputStream fos = new FileOutputStream(file);
    fos.write(target.get());
    fos.flush();
    fos.close();
} catch (Exception e) {
    Log.e("Error", "Saving to external storage failed");
} finally {
    Glide.clear(target);
}

Notes:

I hope you can deploy the finally-clear change and see if it helps that user. Please note that if you're not using the atMost-override-like transformations you're just wasting battery using Glide. There's no point in decoding a Bitmap just to re-encode it again. Glide is an image loading library and you're using it as a simple url downloader. Consider directly using OkHttp or Volley (Glide can integrate with these too), it'll save a lot of power and be much-much faster for the users.

tom-anders commented 8 years ago

Thanks for you detailed answer. I'll try your first suggestion and will look into Volley/OkHttp in the future.

TWiStErRob commented 8 years ago

Let's close it for now, but please report back how it goes after release.

Also make sure you read and understand the end-of-line comments in the second code block!

tom-anders commented 8 years ago

I ended up implementing you second suggestion, but the user reported that it still doesn't work, the crash even happened at the same image. So is there any good tutorial/documentation for using OkHttp or Volley to download images?

TWiStErRob commented 8 years ago

For OkHttp there's a Recipes page, but you can just Google: <library> download file as that's what you want. Here's a good one for OkHttp using their high-level API (second answer on first result).

I think it would be beneficial to figure out the exact reason before or in parallel to transitioning to OkHttp, just in case there's something wrong inside Glide. Can you please help with that?

I have a feeling that it's just too big to fit fully in memory as a Bitmap since you said it crashed exactly at the same image.

tom-anders commented 8 years ago

He's using a Droid Razr on Jelly Bean. This seems to be the image it crashes at:

http://imgs.xkcd.com/comics/online_communities_2_large.png

You're right, it's actually one of the biggest images and his device only has 1GB RAM, so that could be the issue here?

TWiStErRob commented 8 years ago

It's 4MB compressed, the uncompressed in-memory size is around 3072x3571x4=43MB considering that Android application memory varies a lot ranging 16-192MB per app that's an issue.

This is why I included the .atMost().override() in my comments, it makes sure images are no larger than given. Most devices should have enough memory to hold a Bitmap the size of the display so if you constrain to that size you're less likely to run out of memory. By doing so you're using the same amount of network traffic, but pre sizing images like this make sure you can't really crash later even if loading the full images, but you lose details on the image. If you want bigger than screen size you can try estimating the biggest Bitmap you can load with something like sqrt(freeMemory/4) and use that as size.

Using a normal downloader gets around this because it streams the incoming network bytes into a File using less than a MB RAM at most (buffer size).

There's also a Glide downloadOnly API which will give you the File name of what you requested in SOURCE cache, if you really want to stick to Glide.

tom-anders commented 8 years ago

Thanks again for your answer!

Luckily, my app already has an option to disable those large images, so I'll tell the user to try that out. I'll also take a look at the info you provided abut OkHttp.