Jmabrey memory leak - Githubissues

stain commented 9 years ago

An update of jai-imageio/jai-imageio-core#2 from @jmabrey

.. but this breaks the build in all tests:

Tests in error: 
  ConverterTest.testname:32 » IO closed
  Jpeg2000WriteTest.lossless:64 » IO closed
  Jpeg2000WriteTest.lossyWrite:93 » IO closed

Details:

Running com.github.jaiimageio.jpeg2000.Jpeg2000WriteTest
Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 4.249 sec <<< FAILURE! - in com.github.jaiimageio.jpeg2000.Jpeg2000WriteTest
lossless(com.github.jaiimageio.jpeg2000.Jpeg2000WriteTest)  Time elapsed: 2.72 sec  <<< ERROR!
java.io.IOException: closed
    at javax.imageio.stream.ImageInputStreamImpl.checkClosed(ImageInputStreamImpl.java:110)
    at javax.imageio.stream.ImageInputStreamImpl.close(ImageInputStreamImpl.java:857)
    at javax.imageio.stream.FileImageInputStream.close(FileImageInputStream.java:151)
    at javax.imageio.ImageIO.read(ImageIO.java:1451)
    at javax.imageio.ImageIO.read(ImageIO.java:1308)
    at com.github.jaiimageio.jpeg2000.Jpeg2000WriteTest.lossless(Jpeg2000WriteTest.java:64)

lossyWrite(com.github.jaiimageio.jpeg2000.Jpeg2000WriteTest)  Time elapsed: 1.528 sec  <<< ERROR!
java.io.IOException: closed
    at javax.imageio.stream.ImageInputStreamImpl.checkClosed(ImageInputStreamImpl.java:110)
    at javax.imageio.stream.ImageInputStreamImpl.close(ImageInputStreamImpl.java:857)
    at javax.imageio.stream.FileImageInputStream.close(FileImageInputStream.java:151)
    at javax.imageio.ImageIO.read(ImageIO.java:1451)
    at javax.imageio.ImageIO.read(ImageIO.java:1308)
    at com.github.jaiimageio.jpeg2000.Jpeg2000WriteTest.lossyWrite(Jpeg2000WriteTest.java:93)

Running com.github.jaiimageio.jpeg2000.ConverterTest
[, image/png, image/vnd.wap.wbmp, image/jpeg, image/x-portable-graymap, image/bmp, image/pcx, image/gif, image/x-windows-bmp, image/x-windows-pcx, image/x-pc-paintbrush, image/jpeg2000, image/x-bmp, image/x-pcx, image/jp2, image/x-png, image/x-portable-bitmap, image/x-portable-pixmap, image/tiff, image/x-portable-anymap]
[JPEG 2000, JPG, tiff, bmp, PCX, gif, WBMP, PNG, RAW, JPEG, PNM, tif, TIFF, wbmp, jpeg, jpg, JPEG2000, BMP, pcx, GIF, png, raw, pnm, TIF, jpeg2000, jpeg 2000]
[JPG, JPEG 2000, tiff, bmp, PCX, gif, WBMP, PNG, RAW, JPEG, PNM, tif, TIFF, wbmp, jpeg, jpg, JPEG2000, BMP, pcx, GIF, png, raw, pnm, TIF, jpeg2000, jpeg 2000]
/tmp/imageio-test8840333562322252026.JPEG 2000
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.266 sec <<< FAILURE! - in com.github.jaiimageio.jpeg2000.ConverterTest
testname(com.github.jaiimageio.jpeg2000.ConverterTest)  Time elapsed: 0.266 sec  <<< ERROR!
java.io.IOException: closed
    at javax.imageio.stream.ImageInputStreamImpl.checkClosed(ImageInputStreamImpl.java:110)
    at javax.imageio.stream.ImageInputStreamImpl.close(ImageInputStreamImpl.java:857)
    at javax.imageio.stream.FileImageInputStream.close(FileImageInputStream.java:151)
    at javax.imageio.ImageIO.read(ImageIO.java:1451)
    at javax.imageio.ImageIO.read(ImageIO.java:1308)
    at com.github.jaiimageio.jpeg2000.ConverterTest.testname(ConverterTest.java:32)

stain commented 9 years ago

@snoopycrimecop and @mtbc - I see you applied this patch on your fork - but did it not cause similar problems at your end?

Is it possible to formulate a test for this supposed memory leak?

mtbc commented 9 years ago

@melissalinkert: Any conclusions on https://github.com/openmicroscopy/bioformats/pull/1596 ?

mtbc commented 9 years ago

(I'm not sure that we actually ever attempt to write J2K images.)

stain commented 9 years ago

No, this doesn't affect the writing.. but my test writes and then reads, and it fails during ImageIO.read

On 2 March 2015 at 12:22, Mark Carroll notifications@github.com wrote:

(I'm not sure that we actually ever attempt to write J2K images.)

— Reply to this email directly or view it on GitHub https://github.com/jai-imageio/jai-imageio-jpeg2000/pull/2#issuecomment-76702995 .

Stian Soiland-Reyes Apache Taverna (incubating) http://orcid.org/0000-0001-9842-9718

melissalinkert commented 9 years ago

@stain/@mtbc - no, we didn't have this problem with https://github.com/openmicroscopy/bioformats/pull/1596. We do make extensive use of JPEG-2000 writing, but ImageIO.read is never used for reading JPEG-2000 images back - J2KImageReader is always used directly since our use case typically involves very large images for which tile-wise reading is imperative.

It looks like ImageIO.read will try to close the underlying ImageInputStream unless the ImageIO.read(ImageInputStream) signature is used, but that close happens after the J2KImageReader.dispose() call. Calling iis.flush() instead of iis.close() might help, though I'm not sure how well that would fix the memory leak.

stain commented 9 years ago

Thanks for explaining.

Do you have some example (test) code of how you use J2KImageReader that I can use to verify the memory leak?

On 3 March 2015 at 02:28, Melissa Linkert notifications@github.com wrote:

@stain/@mtbc - no, we didn't have this problem with openmicroscopy/bioformats#1596. We do make extensive use of JPEG-2000 writing, but ImageIO.read is never used for reading JPEG-2000 images back - J2KImageReader is always used directly since our use case typically involves very large images for which tile-wise reading is imperative.

It looks like ImageIO.read will try to close the underlying ImageInputStream unless the ImageIO.read(ImageInputStream) signature is used, but that close happens after the J2KImageReader.dispose() call. Calling iis.flush() instead of iis.close() might help, though I'm not sure how well that would fix the memory leak.

— Reply to this email directly or view it on GitHub.

Stian Soiland-Reyes Apache Taverna (incubating), Apache Commons RDF (incubating) http://orcid.org/0000-0001-9842-9718

stain commented 7 years ago

I've not heard anything else, so I'll assume this PR is safe to close unmerged. Please let me know if that is not the case! :-)

jai-imageio / jai-imageio-jpeg2000

Jmabrey memory leak #2