search

Beyka / Android-TiffBitmapFactory

MIT License
131 stars 46 forks source link

Problems converting TIFF to PNG; nodata values copying values from close-by pixels #33

Open GrayCygnus opened 6 years ago

GrayCygnus commented 6 years ago

First, I want to thank you for such a great and useful library :) it is an excellent job.

I just recently needed to convert a TIFF image to PNG, and found that it can be done using TiffConverter.convertTiffPng().

I proceeded to run such method, providing the default ConverterOptions and without listener (null). After the process finishes, the .png image is successfully (no errors thrown, correct height/width) converted, but it is different than the original .tiff image.

The way it is different is that, in the pixels where there should be nodata values (transparent) they seem to "take the pixel value" of nearby pixels. In other words, in a pixel where it should be nodata, it has the value of (say) the pixel N units above it, like it were "filling" the nodata values with the closest chunk or region of pixels closest to the nodata chunk.

Funny analogy: it looks really similar to when you dragged your window on the screen and it "lagged", leaving a trail among the path (which was a copy of one of the edges of the image), on old Windows OS.

Here are some more technical specs (if more are needed will provide them gladly):

Also, did some testing with other images, and found that other, smaller images are converted without this issue. For example:

-Image without issue: 2334x2154, 20.1MB -Image without issue: 3830x3999, 61.3MB -Image with issues: 4862x5065, 99.6MB -Image with issues: 13225x7753, 415.8MB

Both have the same Spatial Reference (WGS84), nodata values, etc., so perhaps the issue has something to do with the size (fails somewhere between 61-99 MB) or the dimensions (perhaps the ones greater than 4096 on either width or height?).

Thanks in advance, and if you need anything else I'll gladly provide.

Beyka commented 6 years ago

Could you please provide some example tiff which converting with this issue? I checked on my test image and all works correctly. Tiff images for test have LZW compression and sizes: 20000x20000 and 13800x11500. File sizes is greater than 300Mb for all.

GrayCygnus commented 6 years ago

Hey @Beyka thanks for the fast reply :D ... I'll check for compression and also will try to provide some sample .tiff. Where should I share it? I'll post it here if possible

GrayCygnus commented 6 years ago

I'm sending a Dropbox link (as it is larger than 10MB) with a sample tiff image that gave me issues (the 99MB one, 4862x5065). Seems that it is a .tiff without compression. I'd like to add that the image is proprietary and has copyright, so I will thank your exclusive use for this testing :)

I'll delete the link after you've got a chance to try it. l'l also add some screenshots depicting the situation described.

GrayCygnus commented 6 years ago

Here we can see on the left, the original image, and on the right the image after conversion (don't know why but today the "lag" is more pronounced than yesterday):

issuewithtiffpng

Beyka commented 6 years ago

You can delete test image from drop box

GrayCygnus commented 6 years ago

@Beyka Done. Will keep posted in case you need anything else, or if I can assist somehow.

GrayCygnus commented 6 years ago

Any luck with the images? @Beyka is there something else I can provide or assist with?

Beyka commented 6 years ago

Sorry, but no luck yet. There some problem with decoding of this image, not only converting.

Beyka commented 6 years ago

@GrayCygnus Could you please check attached build. It will consume more memory, but it uses other decode method and for your images it should work fine( I checked with your test image - it converted ok )

android-tiff.zip

GrayCygnus commented 6 years ago

@Beyka will do, I'll tell you what I get

Beyka commented 6 years ago

@GrayCygnus So, did you test this build?

GrayCygnus commented 6 years ago

Hey @Beyka I haven't had the time to do so, I am currently full with projects. I have scheduled time to test it this week, I'll post when I do so :)

GrayCygnus commented 6 years ago

Hi @Beyka I currently tested the new build you included with the test image (the 99MB one) and it converted without problems! :) Will do some more testing, with that image and others, and will get back to you. Thanks for all your support and sorry for the late reply (much burdened at work, plus some issues in my country).

GrayCygnus commented 6 years ago

Tested with the biggest image I mentioned (13225x7753, 415.8MB) but the app crashed. Also tested with a third image of about 220MB and failed also.

Debugging the error, I see that when the code reaches the TiffConverter.convertTiffPng(inPath, outPath, cvtOptions, null) line, it quickly (almost instantly) finishes it, and proceeds to the next line. However, checking the generated .png file I see it is "empty"... in both instances the result was a 33 B image, which then caused some other error further on the app.

When pressing "Step into" that line of code I see that the TiffConverter class loads 2 libraries inside a static{} context (the native libraries I assume) on lines 46-49:

static {
    System.loadLibrary("tiff");
    System.loadLibrary("tiffconverter");
}

, and then instantly returns to the point I pressed "Step Into"

However, the 99MB image converts without problem, and does stay a while waiting on that line for the conversion to take place. What could be causing these? If you need a new image to test I can provide it.

Beyka commented 6 years ago

Tested with the biggest image I mentioned (13225x7753, 415.8MB) but the app crashed. Also tested with a third image of about 220MB and failed also.

Debugging the error, I see that when the code reaches the TiffConverter.convertTiffPng(inPath, outPath, cvtOptions, null) line, it quickly (almost instantly) finishes it, and proceeds to the next line. However, checking the generated .png file I see it is "empty"... in both instances the result was a 33 B image, which then caused some other error further on the app.

When pressing "Step into" that line of code I see that the TiffConverter class loads 2 libraries inside a static{} context (the native libraries I assume) on lines 46-49:

static {
    System.loadLibrary("tiff");
    System.loadLibrary("tiffconverter");
}

, and then instantly returns to the point I pressed "Step Into"

However, the 99MB image converts without problem, and does stay a while waiting on that line for the conversion to take place. What could be causing these? If you need a new image to test I can provide it.

Try to set TiffBitmapFactory.Options.inAvailableMemory to 512Mb. If converter exit immediately, it's mean that maybe it's not enough memory to process image.

GrayCygnus commented 6 years ago

Try to set TiffBitmapFactory.Options.inAvailableMemory to 512Mb. If converter exit immediately, it's mean that maybe it's not enough memory to process image.

@Beyka I suppose you mean the TiffConverter.ConverterOptions.availableMemory as the error is during conversion, not the one on TiffBitmapFactory.Options.inAvailableMemory? Or did I misunderstood something?

I tried setting TiffConverter.ConverterOptions.availableMemory = 512000000L but also exited instantly. Tried also passing it -1 for "unlimited memory" but also exited immediately... will do some more tests and keep posted.

Beyka commented 6 years ago

Try to set TiffBitmapFactory.Options.inAvailableMemory to 512Mb. If converter exit immediately, it's mean that maybe it's not enough memory to process image.

@Beyka I suppose you mean the TiffConverter.ConverterOptions.availableMemory as the error is during conversion, not the one on TiffBitmapFactory.Options.inAvailableMemory? Or did I misunderstood something?

I tried setting TiffConverter.ConverterOptions.availableMemory = 512000000L but also exited instantly. Tried also passing it -1 for "unlimited memory" but also exited immediately... will do some more tests and keep posted.

Could you send me image that leads to error?

Beyka commented 6 years ago

Try to set TiffBitmapFactory.Options.inAvailableMemory to 512Mb. If converter exit immediately, it's mean that maybe it's not enough memory to process image.

@Beyka I suppose you mean the TiffConverter.ConverterOptions.availableMemory as the error is during conversion, not the one on TiffBitmapFactory.Options.inAvailableMemory? Or did I misunderstood something?

I tried setting TiffConverter.ConverterOptions.availableMemory = 512000000L but also exited instantly. Tried also passing it -1 for "unlimited memory" but also exited immediately... will do some more tests and keep posted.

Could you send me image that leads to error?

@Beyka here is the link to Dropbox where such image is uploaded.

When you manage to download it please ping me so I can delete the link when you are done :)

You can remove link

GrayCygnus commented 5 years ago

Hey @Beyka were you able to test with the new images?

lasat commented 3 years ago

@GrayCygnus Could you please check attached build. It will consume more memory, but it uses other decode method and for your images it should work fine( I checked with your test image - it converted ok )

android-tiff.zip

Hi,

I had the same issue and could acknowledge that the build fix this problem. The TIFF file was generated by ODM.

Any chance that the changed decode methods are integrated as option?

Thank you for this project.