OutOfMemory error for allocating bytebuffer

raylee4204 commented 10 years ago

I'm getting a lot of crashes from my app due to the following error: java.lang.OutOfMemoryError at java.nio.ByteBuffer.allocate(ByteBuffer.java:56) at com.koushikdutta.async.ByteBufferList.obtain(SourceFile:473) at com.koushikdutta.async.http.ResponseCacheMiddleware$BodySpewer.a(SourceFile:410) at com.koushikdutta.async.http.ResponseCacheMiddleware$BodySpewer$1.run(SourceFile:431) at com.koushikdutta.async.AsyncServer.a(SourceFile:675) at com.koushikdutta.async.AsyncServer.c(SourceFile:689) at com.koushikdutta.async.AsyncServer.b(SourceFile:600) at com.koushikdutta.async.AsyncServer.a(SourceFile:37) at com.koushikdutta.async.AsyncServer$13.run(SourceFile:549)

I'm not sure where this happens.

koush commented 10 years ago

Are you downloading really large images?

raylee4204 commented 10 years ago

I actually use another library to download/load images from URLs

koush commented 10 years ago

What are you loading here then?

This is a known bug, but i'm surprised you're hitting it without loading images. Will try to fix it ASAP regardless.

yoavst commented 10 years ago

sorry for the ref, wrong number :(

koush commented 10 years ago

Should be fixed with: https://github.com/koush/AndroidAsync/commit/e62122dcb222999de344517a3be9f4c03016f9c9

Still testing that change though.

ghost commented 10 years ago

I am currently experiencing this bug on a number of different devices and OS versions. The images are not particularly large.

I'm using com.koushikdutta.ion:ion:1.+ so I should have the latest version right?

koush commented 10 years ago

Are you using any transforms? Are they gifs? Are you holding references to too many?

koush commented 10 years ago

Also shoot me the stack trace

ghost commented 10 years ago

I actually realized I was not calling recycle on the original bitmap after applying the transform. This is likely the culprit. I guess I will find out for sure soon enough. On Aug 11, 2014 12:05 PM, "Koushik Dutta" notifications@github.com wrote:

Are you using any transforms? Are they gifs? Are you holding references to too many?

— Reply to this email directly or view it on GitHub https://github.com/koush/ion/issues/266#issuecomment-51825380.

koush commented 10 years ago

Shouldn't need to recycle, that happens automatically on finalize anyways

ghost commented 10 years ago

Here's the stacktrace FYI

java.lang.OutOfMemoryError
       at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:48)
       at java.nio.ReadWriteHeapByteBuffer.<init>(ReadWriteHeapByteBuffer.java:49)
       at java.nio.ByteBuffer.allocate(ByteBuffer.java:52)
       at com.koushikdutta.async.ByteBufferList.obtain(ByteBufferList.java:482)
       at com.koushikdutta.async.ByteBufferList.read(ByteBufferList.java:248)
       at com.koushikdutta.async.ByteBufferList.getAll(ByteBufferList.java:187)
       at com.koushikdutta.ion.LoadBitmap$1.run(LoadBitmap.java:54)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1076)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:569)
       at java.lang.Thread.run(Thread.java:856)

ghost commented 10 years ago

They are not gif all jpeg. Not particularly large.

koush commented 10 years ago

If you can show me the ion call and any transforms I may be able to help.

I'm guessing there's just legitimate memory pressure now, given the new stack trace. This may be an issue with your app and your usage or images rather than ion.

koush commented 10 years ago

That stack is just running out memory trying to load an image

ghost commented 10 years ago

Here are the transforms I'm using

https://gist.github.com/sciencepro/a78350040f224014d507

Unfortunately Crashlytics is making it really hard for me to pin down where in my app this is originating from but all of the Ion calls are pretty standard:

Ion.with(albumImage).transform(roundedTransformationLeft).load(song.albumArtUrl);

and so forth

ghost commented 10 years ago

I remembered that I am logging the image when this happens, all of them are 512x512 images around 120kb a piece.

koush commented 10 years ago

the compressed size is not necessarily relevant, as the uncompressed size is what matters. 512x512 images are fairly large. Roughly 1MB per image in memory.

turbochan commented 9 years ago

Not sure if I should open a new issue or not, but I'm seeing 3 similar variations to this crash. From most prevalent to least:

java.lang.OutOfMemoryError at java.nio.ByteBuffer.allocate(ByteBuffer.java:56) at com.koushikdutta.async.ByteBufferList.java.nio.ByteBuffer obtain(int)(SourceFile:479) at com.koushikdutta.async.ByteBufferList.java.nio.ByteBuffer read(int)(SourceFile:232) at com.koushikdutta.async.ByteBufferList.java.nio.ByteBuffer getAll()(SourceFile:210) at com.koushikdutta.ion.LoadBitmap$1.void run()(SourceFile:54) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587) at java.lang.Thread.run(Thread.java:841)

java.lang.OutOfMemoryError at java.nio.ByteBuffer.allocate(ByteBuffer.java:56) at com.koushikdutta.async.ByteBufferList.java.nio.ByteBuffer obtain(int)(SourceFile:479) at com.koushikdutta.async.util.Allocator.java.nio.ByteBuffer allocate()(SourceFile:24) at com.koushikdutta.async.AsyncNetworkSocket.int onReadable()(SourceFile:153) at com.koushikdutta.async.AsyncServer.void runLoop(com.koushikdutta.async.AsyncServer,com.koushikdutta.async.SelectorWrapper, java.util.PriorityQueue)(SourceFile:766) at com.koushikdutta.async.AsyncServer.com.koushikdutta.async.SelectorWrapper access$300(com.koushikdutta.async.AsyncServer)(S ourceFile:37) com.koushikdutta.async.AsyncServer$ConnectFuture access$500(com.koushikdutta.async.Asyn cServer,java.net.InetSocketAddress,com.koushikdutta.async.callback.ConnectCallback) void access$700(com.koushikdutta.async.AsyncServer,com.koushikdutta.async.SelectorWrapp er,java.util.PriorityQueue) at com.koushikdutta.async.AsyncServer$13.void run()(SourceFile:557)

java.lang.OutOfMemoryError at java.nio.ByteBuffer.allocate(ByteBuffer.java:56) at com.koushikdutta.async.ByteBufferList.java.nio.ByteBuffer obtain(int)(SourceFile:479) at com.koushikdutta.async.util.Allocator.java.nio.ByteBuffer allocate()(SourceFile:24) at com.koushikdutta.async.http.ResponseCacheMiddleware$CachedBodyEmitter.void spewInternal()(SourceFile:414) at com.koushikdutta.async.http.ResponseCacheMiddleware$CachedBodyEmitter$1.void run()(SourceFile:401) at com.koushikdutta.async.AsyncServer.long lockAndRunQueue(com.koushikdutta.async.AsyncServer,java.util.PriorityQueue)(Source File:681) at com.koushikdutta.async.AsyncServer.void runLoop(com.koushikdutta.async.AsyncServer,com.koushikdutta.async.SelectorWrapper, java.util.PriorityQueue)(SourceFile:697) at com.koushikdutta.async.AsyncServer.com.koushikdutta.async.SelectorWrapper access$300(com.koushikdutta.async.AsyncServer)(S ourceFile:37) com.koushikdutta.async.AsyncServer$ConnectFuture access$500(com.koushikdutta.async.Asyn cServer,java.net.InetSocketAddress,com.koushikdutta.async.callback.ConnectCallback) void access$700(com.koushikdutta.async.AsyncServer,com.koushikdutta.async.SelectorWrapp er,java.util.PriorityQueue) at com.koushikdutta.async.AsyncServer$13.void run()(SourceFile:557)

I do load .gifs and various sizes of jpgs: some small and some large. They are all loaded in a ViewPager with only 1 page preloaded on each side.

koush commented 9 years ago

What version of ion are you using @turbochan? Can you point me to some of the images showing this issue?

Can you also paste your Ion call you are using to load the images in the pager?

turbochan commented 9 years ago

Hi @koush , really sorry for the late response. Finally got the data you asked for: I'm using Ion 1.3.8.

Two more issues I'd like to mention: 1) I've also noticed another issue with gifs: a lot of the gifs show some artifacts along the bottom of the image. I've marked two of the offending gifs below with and *. Also here are two unlisted youtube videos of me demonstrating the artifacting: http://youtu.be/4eVcVt5yTS4 http://youtu.be/1g92bcNhE_I I've noticed these artifacts on multiple physical devices (Nexus 5, G3, and from other user reports).

2) Sometimes the gifs or images finish loading but then just present a black screen (or maybe it's no image). Is this what happens if you don't set an error placeholder? I don't see any error outputs from the logcat when this happens though. And I usually can't get the image to show again until I clear the app's data (which probably clears Ion's cache).

Here is a random sample of some images we load, unfortunately I don't have a list of the actual images/gifs causing the OOM crashes that are reported to me through the Play Store. It's hard to tell what the user is looking at since the Play Store crash reports only show the stack trace from the crash.

I do have a user submitted logcat output of a Moto X (1st gen) showing some OOM issues, but I didn't setup the app to log what images are currently loading. Let me know if you would like to see it.

My Ion calls

public Object instantiateItem(ViewGroup container, int position) {
    PhotoView imageView = new PhotoView(mContext);
    imageView.setScaleType(ScaleType.FIT_CENTER);

    String imageUrl = "some image url I fetch at runtime";
    Ion.with(imageView)
        .placeholder(R.drawable.loading_turtle)
        .load(imageUrl);

    container.addView(imageView);
    return imageView;
}

That's pretty much all I do. I forgot to mention I use Chris Bane's PhotoView as the ImageView.

I really appreciate the help Koush. This is the first time I've used your library and I'm relieved that it can let me concentrate on other things other than managing network calls, but I can't seem to squash these OOM and gif artifact issues, which is postponing our staged rollout. Thanks for the response and hopefully this is a resolvable issue.

koush commented 9 years ago

I've opened separate issues to track the GIF problems.

@turbochan is android:largeHeap="true" set? This may help.

How many items is your viewpager loading offscreen? ie, setOffscreenPageLimit? The default is 1, which means that it actually ends up loading 3 images simultaneously. Left, Right, and on screen. Invoking that method with 0 unfortunately does not work. 1 is the minimum.

There's a few problems here, all of which have actually been addressed by Ion 2.0.

You are making "naked" ion calls, no resize/resizeWidth/resizeHeight. Prior to 2.0, this loads the full image, without any scaling. In your case, some of those images are 1024x1024 or higher. Roughly 4MB for an image once decoded. With a ViewPager, you can potentially load 3 very large images and easily end up allocating 12MB+. Depending on what else is going on in your app, this can easily OOM you.

Your video actually shows you loading 4 images (in that grid view), is that in a pager? Would end up loading 12 images total then. Things can get memory constrained quick without that resize call.

Adding some conservative resize arguments would address this in Ion 1.x.

GIFs also exacerbate this situation. Since, prior to Ion 2.0, every frame of the GIF is loaded up front. GIFs can easily end up being 20MB+ RAM once fully decoded.

There's no fix for this in Ion 1.x.

Ion 2.x handles memory much better with regards to ImageViews. Images are fetched and decoded only at draw time (vs upon Ion invocation). The ImageView bounds are then used provide an intelligent sampleSize on decode to reduce memory usage.

Ion 2.x also has progressive GIF decoding, so only one frame is loaded at a time. This reduces memory an insane amount... 30x less memory used if its a 30 frame GIF.

Long story short, try ion 2.0 out if you're memory constrained. I've pushed it out on all my apps already. https://plus.google.com/110558071969009568835/posts/EXS5UrMa49X

turbochan commented 9 years ago

I do not have android:largeHeap set in my manifest. I was actually unaware of this; thanks for pointing it out. I'll let some users try that out.

My ViewPager loads 1 item to each side, so 3 at a time like you said.

I think I'm slightly confused about the 4MB image size you mention. Does the size of the actual files (most of the ones I linked seem to be in the 100-400KB range) differ from what they are when Android displays them ("decodes" them)?

The GridView you see in the video is indead within a ViewPager. But I don't use Ion to populate that thumbnails ViewPager. I use some old school code with plain AsyncTasks for that and my own simple cache. They also load some pretty low res thumbnails, different than the images I linked up above. That was before I discovered Ion. After you tap on a thumbnail it opens a new Activity that has it's own ViewPager where I actually use Ion to load the images (with 1 page on each side).

I was hoping to avoid having to resize the images since some images need zooming to allow the user to actually see the details. For example, I think this tall image would be impossible to read if I resized it to a smaller size: http://cdn.funnyisms.com/b57102e9-a858-4b2d-892b-32a0b6e2fd32.jpg

If I use the resize() call with some conservative sizes, say 512 on the longest length) and turn on deepZoom, would that be a possible solution? Can the user still zoom in and see extra details get loaded? Or would the resize() basically fix the image, not allowing any new data to load upon zoom?

Thanks so much for the help! Also is Ion 2.0 out? Or are you alluding to a future release?

koush commented 9 years ago

@turbochan if you need to keep the high resolution images, I highly recommend using .deepZoom, which was built with this in mind. The demo actually uses PhotoView. Large images will have a small memory footprint.

https://www.youtube.com/watch?v=yIMltNEAKZY

In that video, I loaded a 9,865 × 3,120 pixel image.

Ion 2 is already released. Just update your gradle/maven/whatever reference.

koush commented 9 years ago

It's also worth keeping in mind that OutOfMemoryError is all luck of the draw. Ion may be doing its best to keep memory usage low, but using another different image loading mechanism could use up all the memory, and cause anywhere else in the application to fail. It may be a String allocation, or it may be Ion.

I'd recommend switching all your image loading to Ion. Ion can't purge images from memory that are loaded via other means...

koush commented 9 years ago

The 4MB size is how big a 1024x1024 image is in memory after it is decoded. The image may be 70k compressed, but loaded as an argb8888 Bitmap, it will be 4MB.

turbochan commented 9 years ago

Thanks a bunch. I was definitely planning to move all my image loading to Ion, as well as a lot of my other networking calls when I got a chance.

.deepZoom isn't meant to be used with the resize() call correct? Looking at your DeepZoomSample app I don't see a resize() call. Also is it safe to leave it on for all images, even small ones, as a user pages through them?

~~When was 2.0 released? I thought I just downloaded these jars in Oct.~~ (EDIT: I see the changelog now) I'm trying to download the jar (for androidasync.jar as well) at https://github.com/koush/ion#get-ion but it's asking for authentication. Unfortunately I have yet to learn how to use Maven/Gradle so I usually just manually drop jars into \libs in my Android projects.

koush commented 9 years ago

Correct, deepzoom an resize won't work together.

koush commented 9 years ago

It is safe to leave on for all images.

koush commented 9 years ago

Yeah, there's an another issue on the sonatype download... not sure why it is asking for auth now. Can grab androidasync and ion from maven central though.

http://search.maven.org/#search%7Cga%7C1%7Ckoushikdutta

turbochan commented 9 years ago

Awesome, I just got them. I'll give them a shot.

koush / ion

OutOfMemory error for allocating bytebuffer #266