search

coolshou / thumbnailator

Automatically exported from code.google.com/p/thumbnailator
Other
0 stars 0 forks source link

OutOfMemoryErrors (memory leak) when using latest versions of Java 6 and 7 #42

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
1. Unzip the attached maven project
2. cd thumbnailator-test
3. mvn test 

Maven runs the test with -Xms256m -Xmx256m. It resizes a png file 100 times, 
then resizes a jpg around 40 times before throwing an OOME. You can also build 
and run an executable jar with 
mvn -DskipTests=true package
java -Xms256m -Xmx256m -jar target/jpegleaker.jar

This fails on Ubuntu, java-version
java version "1.7.0_21"
Java(TM) SE Runtime Environment (build 1.7.0_21-b11)
Java HotSpot(TM) 64-Bit Server VM (build 23.21-b01, mixed mode)

This works on Ubuntu java -version
java version "1.7.0_17"
Java(TM) SE Runtime Environment (build 1.7.0_17-b02)
Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode)

Original issue reported on code.google.com by will.t...@xtremelabs.com on 16 May 2013 at 12:58

Attachments:

GoogleCodeExporter commented 9 years ago 
After some investigation it looks like dispose() needs to be called on 
ImageReaders and ImageWriters after using them. The methods affected would be 
net.coobird.thumbnailator.tasks.io.InputStreamImageSource.read()
net.coobird.thumbnailator.tasks.io.OutputStreamImageSink.write(BufferedImage)
net.coobird.thumbnailator.tasks.io.FileImageSource.read()
net.coobird.thumbnailator.tasks.io.FileImageSink.write(BufferedImage)

Original comment by will.t...@xtremelabs.com on 16 May 2013 at 2:19

GoogleCodeExporter commented 9 years ago 
Hi,

Thank you for taking your time to report this issue.

It does appear like there are many instances of people encountering 
`OutOfMemoryError`s when using Thumbnailator with Java 7 Update 21.

The need to address this issue has been acknowledged.

Original comment by coobird...@gmail.com on 20 May 2013 at 1:47

GoogleCodeExporter commented 9 years ago 
Issue 44 appears to be the same issue.

Original comment by coobird...@gmail.com on 20 May 2013 at 1:48

GoogleCodeExporter commented 9 years ago 
I've also gotten a report of a memory leak via an e-mail from Vladimir Shomin 
on April 24.

Quote:

> I have discovered some memory problem due exploiting system, when thousands of
> images have queued for thumbnailation - application server tomcat-7.0.39
> under jdk-1.7.0_21 crushes with OutOfMemoryError.
>
> Exploring heap state with VisualVM tool, I have found that memory is filled
> by megabyte-sized byte[] objects that linked as root by
> com.sun...JPEGImageReader . Forums recommend to use BufferedImage.dispose()
> and ImageReader.flush() methods beyond image processing cause its releases
> low-level resources used within image manipulations.

Again, this appears to occur when using Java 7 Update 21, which is the most 
up-to-date version of Java available as of the time of writing this comment.

Therefore, this issue should be given the highest priority.

Original comment by coobird...@gmail.com on 20 May 2013 at 1:59

GoogleCodeExporter commented 9 years ago 
Issue 44 has been merged into this issue.

Original comment by coobird...@gmail.com on 20 May 2013 at 2:22

GoogleCodeExporter commented 9 years ago 
Reproducing the Problem
-------------------------

In an effort to reproduce the problem, I ran a variation of the following code:

  Thumbnails.of(new File("/path/to/images").listFiles()
    .scale(0.1)
    .toFiles(new ConsecutivelyNumberedFilenames(new File("/path/for/thumbnails")));

I set the JVM's maximum heap settings to `-Xmx96m`, as that was the easiest 
value to see the Java 1.7.0_21 JVM to throw an `OutOfMemoryError` while 
processing images.

Using Thumbnailator 0.4.3 with the above settings, Java 1.7.0_21 would throw an 
`OutOfMemoryError` after processing 15 JPEGs which had the dimensions of 3648 x 
2736 pixels.

In contrast, when running with the same settings on Java 1.7.0_4, an 
`OutOfMemoryError` did not occur.

Attempting a Fix
-------------------------

As pointed out by the issue reporters, the problem probably lies in the lack of 
disposal code for the `ImageReader`s and `ImageWriter`s.

First, the disposal of the `ImageReader`s in the `FileImageSource` and 
`InputStreamImageSource` classes were added.
However, after running the test code, it encountered an `OutOfMemoryError` at 
15 images; the same as last time.

Next, the disposal of the `ImageWriters`s in the `FileImageSink` and 
`InputStreamImageSink` classes were added.

The results this time was that the test code would run without encountered an 
`OutOfMemoryError`.
(The code was subsequently changed to output to PNG, and the results were the 
same -- no `OutOfMemoryError`.)

As an added measure, a call to `BufferedImage.flush` has been added to the 
`Thumbnailator.createThumbnail(ThumbnailTask<?, ?>)` method, which should free 
up some resources as well.

Current Status
-------------------------

Thumbnailator 0.4.4 which addresses this `OutOfMemoryError` issue should be 
ready (hopefully) by the end of the week.

The changes has been pushed to the repository under the `0.4.4` branch:
http://code.google.com/p/thumbnailator/source/browse/?name=0.4.4

All unit tests are passing at this point, but I'll perform some manual testing 
before release.

Original comment by coobird...@gmail.com on 21 May 2013 at 4:16

GoogleCodeExporter commented 9 years ago 
Issue 46 is similar issue (though that may not mean the cause is similar). 
Please take a look as well.

Original comment by jain...@gmail.com on 22 May 2013 at 3:33

GoogleCodeExporter commented 9 years ago 
Issue 46 is for jdk 1.6 build 45.

Original comment by jain...@gmail.com on 22 May 2013 at 3:34

GoogleCodeExporter commented 9 years ago 
I have reproduced the problem with Thumbnailator 0.4.3 running on Java 6 Update 
45.

When running the candidate for Thumbnailator 0.4.4 on Java 6 Update 45, it does 
not exhibit the `OutOfMemoryError` problem.

I'll merge Issue 46 into this issue.

Original comment by coobird...@gmail.com on 23 May 2013 at 12:23

GoogleCodeExporter commented 9 years ago 
Issue 46 has been merged into this issue.

Original comment by coobird...@gmail.com on 23 May 2013 at 12:25

GoogleCodeExporter commented 9 years ago 
Issue 45 has been merged into this issue.

Original comment by coobird...@gmail.com on 23 May 2013 at 2:45

GoogleCodeExporter commented 9 years ago 
This issue has been resolved in Thumbnailator 0.4.4.

I'd like to thank everyone involved in resolving this issue for contributing 
your time and effort! :)

Original comment by coobird...@gmail.com on 23 May 2013 at 3:49

GoogleCodeExporter commented 9 years ago 
was this a bug in a particular jdk version? was there a commit that fixed it? 
just wondering

Original comment by rogerpack2005 on 7 Jan 2015 at 12:19

GoogleCodeExporter commented 9 years ago 
No bug in the jdk, just a bug in Thumbnailator where resources weren't being 
cleaned up. It only seemed to matter in more recent jdk versions which were 
doing some optimizations by re-using resources (assuming they'd be disposed of 
when necessary)

Original comment by wt...@pivotal.io on 7 Jan 2015 at 12:39