search

dragon66 / icafe

Java library for reading, writing, converting and manipulating images and metadata
Eclipse Public License 1.0
203 stars 58 forks source link

Java out of memory exception #73

Closed suncds-git closed 6 years ago

suncds-git commented 6 years ago

When reading/writing images for large tiff files, receiving Out of Memory exception error. This is occurring whenever multiple tiffs are being read, or written - TiffReader.read() TiffTweaker.insert pages() and TIFFTweaker.writeMultipageTIFF(. I am working with some rather large multi page tiff files. Is there a way to read/write a single tiff page at a time, and also append to existing tiff file?

dragon66 commented 6 years ago

For inserting pages to existing Tiff or writing multiple pages Tiff, there are methods available to handle pages one at a time. For inserting, the method combination is prepareForInsert(), insertPage()(called multiple times until all pages are inserted), and finishInsert() to wrap up.

For writing multiple pages Tiff, the corresponding methods are prepareForWrite(), writePage()(called mutilple times until all pages are written), and finishWrite().

For append pages one by one, there is no methods can be used directly but you can do the same thing as you do with insert pages. The trick to use is every time you call insertPage(), you need to pass the list.size() (here list is a list of IFDs passed as parameter) as the page number for the insertPage() method. This will ensure the page be appended at the end.

You can refer to the latest TIFFTweaker source code where I updated the comments recently for the corresponding methods with examples.

For reading Tiff, the current implementation is to read all pages and return the first page. This may become an issue if the image is extremely large. Although there is code to read GIF image frame by frame, there is no ready to use one for Tiff. Some code change needs to be done to achieve similar result.

Hope this helps a bit.

Wen

suncds-git commented 6 years ago

Thanks, Wen for your quick reply. I am reviewing the TiffReader.read() method, This method is reading and storing all the images in the method's scope, which is causing an out of memory error. You are correct that the first page is being returned from a List object. I believe that I do need the capability of reading one tiff image at a time, so that the image may be read and then appended to the existing tiff file.

dragon66 commented 6 years ago

I pushed some changes to address the read issue. A new class called PageReader is created which is a sub-class of TIFFReader to address the requirement of reading multipage tiff one page at a time. It is similar to what I did for animated GIF where frames could be read one at a time.

A test case called TestPageReader is also added along with PageReader. These changes should address the memory issue. Unfortunately, TIFFReader itself is not a full-fledged one especially support for CCITT FAX (by which most of the multipage Tiff images are compressed) is not completely done yet.

So it may not help you a lot at the moment until TIFFReader is done.

Wen