Get corners of geotiff

[kmbisset89]

Is there a way to get the corners of the geo tiff from the metadata? I see location but I assuming that is the center point.

[StefanOltmann]

Regrettably, I don't possess a sample file containing a GeoTiff for reference in that particular subject. Would it be possible for you to share one with me instead?

[mipastgt]

Here is the standard document which explains it all. (Very long but it is actually not so difficult) https://gis-lab.info/docs/geotiff-1.8.2.pdf A simpler version which focuses on what is needed. http://geotiff.maptools.org/spec/geotiff2.6.html Actually you do not even have to understand the meaning of all the values because you just want to read and write them.

I am just working on this myself. See https://github.com/Akaflieg-Freiburg/enroute/issues/391

Attached you find two example files which use two different variants to describe the geo-reference information. The first uses one tie-point + pixel scaling and the second one uses a full affine transform matrix.

EDDS_Stuttgart 3-geo.tiff.zip EDDS_Stuttgart 5-geo.tiff.zip

[StefanOltmann]

@mipastgt Thanks, very helpful.

I found a comprehensive list of tags listed here: https://exiftool.org/TagNames/GeoTiff.html

https://stefan-oltmann.de/exif-viewer shows the tags in the file, but as "Unknown".

Using low level API it's possible to read & write these values using Kim, but I think I can provide a more convenient API to do so.

[mipastgt]

Maybe this helps too. It works with recent standard Java (which includes ImageIO tiff-support by default) but it won't work on Android for example. I extracted that from Degree and converted it to Kotlin. ImageIOGeoTiffWriter.zip

[StefanOltmann]

@mipastgt Thank you for your input!

Kim v0.17 now recognizes the GeoTiff tags and reports them.

Also a GeoTiffDirectory object is parsed from the ShortArray in the GeoKeyDirectoryTag (0x87af). It's now part of TiffContents, which is part of ImageMetadata.

The read support is limited to the fields I found in your provided sample files. A full implementation of the whole standard will take its time - similar to the planned full MakerNote support. I focused on the fields that seem actually be useful for you right now.

As you requested I added a line "GeoTiff support is limited and supports only reading at this time." to the "limitation" section. What I mean by this is that right now there is not a convenient high-level API to write the tags similar to the tooling you wrote in the attached source code.

But using the low level API we inherited from Apache Commons Imaging you actually can add the tags to any support file format right now in a similar way you did this for the other API. The low level part of the README explains this.

TiffOutputDirectory has many add() methods like this one. If a certain TagInfo is missing it can be created locally. Apache Commons Imaging has this nice design that allows extensibility which we kept for this reason.

So writing should be possible since the first version, but we can still make it more comfortable. If you should try this out for yourself, we are of course always happy over a PR. :)

[StefanOltmann]

@kmbisset89 Can you explain in more detail what you mean by "corners of the geo tiff"? Which field is that? I'm happy to show you how to read that using this library.

@mipastgt Do you have an idea what this means?

[mipastgt]

I can only guess what he/she means but I think he/she is interested in the geo-locations of the corners of the image. These could be computed by the values stored in the TIFF tags. These values define the mapping from pixel coordinates to world coordinates and so you could compute the world coordinates of the corner coordinates (0,0), (0,height), (width,height) and (width,0). But I think that is beyond the scope of a metadata library and is the job of the user of the data.

In the general case this would actually be not so easy because you cannot just assume that the CRS is always 4326 (https://epsg.io/4326) as I assumed in my examples. In general that could be any system and you would have to have the whole GIS machinery at hand to compute what you want.

[mipastgt]

I probably don't understand the low-level API of Kim. Could you provide me with an example of how to write the GeoTIFF metadata into an existing TIFF file which doesn't yet contain such metadata? Don't care for the correct values. I just want to understand the API.

[StefanOltmann]

into an existing TIFF file

Unfortunately that's not possible. I checked the sources, but I stripped away the ability to modify existing TIFF files. The existing TiffWriter can only create new TIFFs, because EXIF metadata is TIFF based.

I think Apache Commons Imaging is capable of that and I can look into what it takes to restore that logic.

Ashampoo Photos doesn't update TIFF files, because most of the RAW formats are based on TIFF and half of them can’t even be distinguished from regular TIFF. As you never want to touch/update RAW files, this ability is also somewhat dangerous.

Another issue is of course that SKIA does not support TIFF. So even if I was able to tell RAWs and regular TIFFs apart, Ashampoo Photos won't be able to display them. That's why there was no need for TIFF so far.

Is TIFF essential to you? Or would WebP or PNG also work?

I added a sample how to add GeoTiff to an JPG. The same would also work for all other formats Kim has write support for.

[mipastgt]

I support TIFF in my software because I was asked to do so and because it seems to be the only image format which directly supports the embedding of geo-reference data. But I do not really like it and would prefer to abandon it as soon as possible if I could find a reasonable altenative. I am currently investigating possible alternatives but haven't yet finished that.

I tried your example and it actually worked but when I tried to display the resulting file in QGIS, it did not recognize the geo data. I assume that other software like QGIS just doesn't expect to find this TIFF data in a JPEG file and thus doesn't try to read it.

For the time being I'll focus on the alternatives like, e.g., associated ESRI world files.

[StefanOltmann]

@mipastgt Can you attach a sample file how your TIFF files look without the metadata?

[mipastgt]

I don't have one at hand at the moment but shouldn't this

        // Create a copy of the input file without metadata.
        val inputImage = ImageIO.read(inputFile)
        ImageIO.write(inputImage, "tiff", outputFile)

remove all metadata from an existing image file with metadata?

[StefanOltmann]

Yes, that should. I just wanted to make sure to test a sample file that looks exactly like what you have in your pipeline at the moment where Kim would come into action.

[mipastgt]

Actually that is the pipeline. I get larger images as PNGs, crop them a bit and write them out as TIFFs. On the JVM this would be the exact code as above if I exclude cropping and adding the geo-info.

[StefanOltmann]

@mipastgt Just for you I released v0.17.1 which preserves image data in tiff files. You should now be able to add your GeoTiff to an existing tiff. See the this example how to do that.

[mipastgt]

Thanks a lot. Just in order to avoid misunderstandings on my side. This feature makes it possible to decouple the process of creating/writing the image from the process of adding this geo-metadata to the image, right? So I can use whatever library or built in feature I can find on any platform to create the image and then use Kim in my common code to add the metadata? Is that assumption true?

[StefanOltmann]

@mipastgt Yes, it should work like that. I used a TIFF created by GIMP, but any valid TIFF should work. If you find one that does not work, share it with me and I will take a look at it.

So you can create your images in a platform-specific way (since skiko can't create TIFF images) and add your GeoTiff using Kim in shared code across all platforms.

Let me know if it works for you. :)

[mipastgt]

One more question. You showed me (and I tested it) that you can also write this geo-metadata into any image (PNG, JPEG, WEBP, TIFF). The problem with this was that most software cannot handle that because it simply does not expect to find this data in there. Some of the people I work with are using C++ and Qt to write their software for desktop, Android and iOS. Do you know any cross-platform C++ library which would allow to extract this geo-metadata from any non-TIFF image file? I am not going to dive into that but I could forward them any hint you have.

[StefanOltmann]

The only C++ metadata library I know of is exiv2 and that one is under GPL license.

Can native C++ code use XCFrameworks? In that case for iOS & macOS the kim.xcframework might be an option.

For native Windows, macOS & Linux there are native libraries. I was only able to verify the native executables so far. The libraries are created just because it's possible. It would be nice to hear from C developers if they actually work.

See https://github.com/Ashampoo/kim/releases/tag/v0.17.1

Native Android libraries could also be supported. It's just a matter of adding those targets.

[mipastgt]

Concerning your example from above. How would that look like in a multiplatform scenario? I was able to replace the upper part by

        val inputFile = "empty.tif".toPath()
        val outputFile = "geotiff.tif".toPath()

        val inputBytes = SystemFileSystem.read(inputFile) { readByteArray() }
        val metadata = Kim.readMetadata(inputBytes) ?: return

but what about the lower part where you write the new metadata to the existing output file. I used Okio here but I think kotlinx-io would work the same.

[StefanOltmann]

You need to provide a ByteWriter, which is a pretty simple interface.

A very basic multiplatform implementation of that is ByteArrayByteWriter, which will produce a Kotlin ByteArray as the name suggests. For kotlinx-io you can use KotlinIoSinkByteWriter. You can of course also write your own implementation for Okio.

Note that Kim can directly read from kotlinx-io Path: https://github.com/Ashampoo/kim/blob/e1f8ce8ebe05ca96c52492ce24b59fd3cbbfa478/src/ktorMain/kotlin/com/ashampoo/kim/Kim.ktor.kt#L38-L43

That's more efficient, because Kim is designed to only read the header of the files for metadata instead of the whole file. TIFF files can be very large. We have an optimized reader for that.

Another hint: If performance is crucial and you now that you exclusively deal with TIFF files you could also call TiffImageParser.parseMetadata() directly and skip the file format detection step.

[StefanOltmann]

I wrote more sample code for you, but I ran into a weird IDEA import solving problem. I work on a fix for that right now.

[mipastgt]

I am currently back to square one. Based on your example above I came up with the attached full example. It runs without exception but has several problems though. The main problem is that the resulting file just has a few kilobytes and doesn't seem to contain anything else but the metadata. (The format of the files is big-endian by the way.) The original files are also lossless compressed (Compression 8). But the lossless TIFF writer has an aditional constructor parameter which I don't understand. So, in the end I was not able to reproduce the orginal files. Maybe you can shed some more light on this. KimTiffGeoWriteTIFFJVMTest.kt.zip

PS: I will only be able to respond occasionally from now on because I will be attending the JavaLand conference at the Nürburgring until end of next week and I still have to prepare a lot for this trip.

[StefanOltmann]

If you update to Kim v0.17.2 this should be fixed. The TIFF produced by ImageIO has multiple strip bytes - this is now also supported.

The lossless version of the TiffWriter has nothing to do with the image compression. That one does extra steps to preserve Makernote field offsets. That's important, because otherwise this part of the EXIF data will get corrupted and useless. If you don't have Makernote in your TIFFs you can safely ignore that. If you use Kim.update() API on images it will automatically detect which version of the writer it needs to use.

AFAIK ImageIO.write() should be a synchronous call, but in my test the image was not fully written and resulted in a black image. After adding a short delay this worked as expected.

Here is a updated sample that suits your case: https://github.com/Ashampoo/kim/blob/9959eee74fd255fe2afcb456192e9a84ff1484c8/examples/kim-kotlin-jvm-sample/src/main/kotlin/Main.kt#L214-L269

I wish you a lot of fun at JavaLand! :)

[mipastgt]

I ran a test with your new library, your sample code and my sample TIFFs. The result is the following:

1. TIFF output is created.
2. Contains geo-metadata.
3. Other software (tested with QGIS) understands the TIFF and shows it correctly on a map.

But:

1. The output TIFF is twice as large as the original (18.3MB).
2. There are differences in the metadata which may indicate a problem. Some of them look strange.

Original image:

0000000046 0x0103 Compression = 8
0000000082 0x0111 PreviewImageStart = [344 ints]
0000000106 0x0116 RowsPerStrip = 8
0000000118 0x0117 PreviewImageLength = [344 ints]

Updated image:

0000000046 0x0103 Compression = 1
0000000070 0x0111 PreviewImageStart = 336
0000000082 0x0112 Orientation = 1
0000000106 0x0116 RowsPerStrip = 2147483647
0000000118 0x0117 PreviewImageLength = 18344466

[StefanOltmann]

Great to hear that it works for you! :)

The output TIFF is twice as large as the original (18.3MB).

Yes, that's an effect of rewriting the image using ImageIO:

val inputImage = ImageIO.read(inputFile)
ImageIO.write(inputImage, "tiff", tempFile)

That writes uncompressed image bytes, as indicated by 0x0103 Compression = 1. Try to use the same compression level.

0000000082 0x0112 Orientation = 1

By default Kim adds an TIFF orientation flag where missing. This enables changing the orientation by just flipping one byte.

0000000070 0x0111 PreviewImageStart = 336 0000000118 0x0117 PreviewImageLength = 18344466

The TIFF produced by ImageIO has multiple image strip elements while my file produced by GIMP has just one. That's the thing I fixed in v0.17.2. I now detect them all and merge them together. That's way easier to handle in rewriting the file.

As the image is correctly shown this makes no difference.

You can inspect both files using the HEX view of https://stefan-oltmann.de/exif-viewer for a better understanding.

I just re-saved your file using GIMP and saw that GIMP changes the strips. They are now chunks of 128, while your original has chunks of 8. I'm actually not sure what a good number is.

Google says this on strips: The number of rows per strip. TIFF image data can be organized into strips for faster random access and efficient I/O buffering. RowsPerStrip and ImageLength together tell us the number of strips in the entire image.

A possible future improvement can be to rewrite the file using the same strip count. That's some work, but it also means less things changed. This would be similar to how Kim preserves the byte order.

0000000106 0x0116 RowsPerStrip = 2147483647

Yes, that's Int.MAX_VALUE. I'm sure there is logic to calculate the correct value after merging the strips, but Apache Commons Imaging uses this value as default and it works. Tell me if you find a reader that doesn't like this and I will investigate how to calculate the correct number.

[mipastgt]

OK, the size was my fault. I forgot that I did not use the defaults in my original code. The rest is explainable. So we are done. Thanks a lot for your patience and help :-)

[StefanOltmann]

So we are done.

Not yet. There is a update underway that needs you to change a small thing for the next version:

https://github.com/Ashampoo/kim/blob/9ab5fb627d3d450286c94d1d7bb0f3335629c6c2/examples/kim-kotlin-jvm-sample/src/main/kotlin/Main.kt#L201-L205

I noticed that I can't read the strip bytes by default. This slows down Ashampoo Photos on RAW metadata reading to much. So strip bytes should only be read if they are needed to rewrite a TIFF.

v0.17.3 should become available on Maven Central in the next hours.

Thanks a lot for your patience and help :-)

You're welcome! I'm always happy to help fellow developers, and I hope that Kim proves to be valuable to you. Feel free to spread the word about it. I strive to make it as useful as possible, so it garners plenty of GitHub stars. ;)

[mipastgt]

I still have a problem with some TIFFs. After adding the metadata like above the resulting TIFF is broken and cannot be displayed anymore. When I add the TIFF to QGIS it shows the following messages:

EDKA_Aachen-Merzbrueck 1-geo: /Users/mpaus/AIPBrowserDE-Export/Android/EDKA_Aachen-Merzbrueck 1-geo.tiff, band 1: IReadBlock failed at X offset 0, Y offset 0: TIFFReadEncodedStrip() failed.
EDKA_Aachen-Merzbrueck 1-geo: /Users/mpaus/AIPBrowserDE-Export/Android/EDKA_Aachen-Merzbrueck 1-geo.tiff, band 2: IReadBlock failed at X offset 0, Y offset 0: TIFFReadEncodedStrip() failed.
EDKA_Aachen-Merzbrueck 1-geo: /Users/mpaus/AIPBrowserDE-Export/Android/EDKA_Aachen-Merzbrueck 1-geo.tiff, band 3: IReadBlock failed at X offset 0, Y offset 0: TIFFReadEncodedStrip() failed.

I have attached the original tiff and the geo tiff. The original tiff was created by some other TIFF writer on Android and not via ImageIO on desktop as before. The only major difference I can see is that the original TIFF seems to have an alpha channel and the previous examples did not. Maybe you can find out what the problem is here.

two_images.zip

[StefanOltmann]

Yes, the file seems to be corrupted after merging the strips bytes. But I don't see any obvious hint what could be wrong here.

Let me know if you find a tool that can correct the file or explain what's wrong. That may show us where the problem is.

The QGIS output does not really help me here.

[StefanOltmann]

@mipastgt

I figured out what the issue is. This file is compressed. It would make sense if the individual strip segments are compressed and cannot be merged together.

Using https://stefan-oltmann.de/exif-viewer I noticed that they all start with 78 9C, which is the ZIP header.

For now you can't use compressed TIFF until I figured out how to preserve all strip chunks.

[mipastgt]

I can confirm that the file is readable without compression. Also the geo-reference data seems to be correct according to QGIS. The size is a problem though. It's now 8.7 MB instead of 885 KB. I'll try some compression schemes in order to see whether they make a difference.

[StefanOltmann]

I understand your point. I experimented a bit, but the current logic doesn't support updating fields with multiple offsets, only one at a time. I've spent some time exploring ways to protect the fields similar to how it's done for the Makernote, but it's proving to be quite challenging.

Initially, TIFF writing wasn't part of the plan, so altering the design now poses some complications.

Are you aware of any libraries capable of compressing TIFF files without discarding metadata, essentially by ignoring unknown fields?

[StefanOltmann]

It's about time for GeoPNG. ;)

[mipastgt]

I tested compression NONE, ADOBE_DEFLATE, LZW and JPEG. From these only NONE and JPEG worked. I can live with JPEG for the moment. The file size is not super small but with 1.2 MB it is at least substantially smaller than NONE.

Concerning your question - no I don't know any such library.

I'd say GeoWEBP.;)

[StefanOltmann]

I believe ADOBE_DEFLATE is basically the same as ZLIB.

For PNG support Kim comes with zlib support. So it would be possible to decompress the strip bytes, merge them and compress them again. But this is also some work.

I'm glad to hear that the JPEG strip bytes can be merged together without getting corrupted.

I like PNG more than WebP - it's simpler. :)