Add support for TIFF image format

[fire]

Describe the project you are working on

Write a converter for an industry standard game engine format so we can import into godot engine for transferring existing assets.

Describe the problem or limitation you are having in your project

To import a tiff one has to use imagemagick's convert.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

If Godot Engine has direct support for tiff, it would not be necessary to use convert.exe.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

Use https://github.com/syoyo/tinydng 's tiff implementation to write a tiff importer.

If this enhancement will not be used often, can it be worked around with a few lines of script?

Not a few lines of script.

Is there a reason why this should be core and not an add-on in the asset library?

Unsure. Polling for interest.

[Calinou]

How common is TIFF in glTF/FBX game scenes? I rarely see TGA or BMP used there in the first place.

[fire]

I'm not sure. Leaving this for others as I can code this straightforwardly.

[Zireael07]

@Calinou Fairly rare but from time to time I do see TIFF used on OpenGameArt and the like.

[MarioLiebisch]

Think I've actually seen TIFF being used for some officially published heightmap data sets (e.g. GIS stuff) and the like, so in theory there might be a use for this in more science related projects.

[Sythelux]

I'm currently working on a scientifcal GIS-Project and yes indeed it is very common to use tiff especially for height maps. there is also an extra GeoTIFF Subformat for map-projection. Latter is a very small usecase, but it would be good to have the API.

But pretty much every game-developer that wants to use realistic heightmaps finds tiff files first.

[DeeJayLSP]

Despite being much more complex, I think libtiff is a better option since it supports many common compression algorithms, which include PackBits (the default when converting an image with FFmpeg), LZW and DEFLATE. From those three, TinyDNG seems to support only LZW. Unless some kind of separate implementation is done to force it to include those algorithms.

[fire]

@DeeJayLSP Can you write a proposal for PackBits on https://github.com/syoyo/tinydng? It's worth a try.

[ghost]

Hi

@fire do you still want to add TIFF support? I'm using a lot of models which are originally W3D models, converted to OBJ format, that use TIFF for textures. So I now need to convert all textures manually. It would be awesome if Godot supports TIFF Format!

You can find them here (click on "models"):

https://www.mameworld.info/3darcade/

I would LOVE to have TIFF support and as @DeeJayLSP says, maybe LibTIFF is better?

https://libtiff.gitlab.io/libtiff/

Thanks in advance!

[fire]

The feature proposal window is closed on Godot Engine 4.0 since July. There may be a few exceptions, but we're trying to fix bugs to release a new version. I do spend my Fridays experimenting, but no promises.

I am not so positive about adding libtiff natively, so I was trying to find a lighter version.

OPENEXR handles the hdr (with layers and abitrary count of channels).

WEBP handles regular images with lossless and loss. PNGs have a problem where rgba's alpha channel is confused with a different feature in photoshop. WEBP has a problem where we cannot assume it is lossless on creation in larger teams.

In my humble opinion, I don't see a spot for it in Godot Engine's import pipeline that is essential. You can see how openexr and webp are providing essential features.

[MichaelMacha]

Looking back at my old experiments in terrain generation outside of Godot, the one thing I ended up loving TIFF for was its ability to be reliably lossless and reasonably flexible; which is also why it's the basis for GeoTIFF for geography. That said, it's also a very old format and has largely been superseded by things like EXR, which are more modernized and are already supported by Godot... and then there's the little issue of the god-killer file sizes for TIFF.

However, if I'm using something like a texture off of Textures.com, there's a very real chance it's initially going to be in TIFF format. It's still extremely popular among photographers, and a lot of the shooters who stopped using it have unfortunately gone to PSD (which I cannot realistically advocate any more than FBX, for the same reasons).

I feel like it would be a cool thing to have for speed purposes, but it's also not-that-difficult to just convert it in imagemagick before I use it, probably to EXR. I'm more concerned for the people who wouldn't initially think of doing that. It might be easy, but it isn't "entry level".

I'm still not sure how to do it myself, but I wouldn't mind if somebody could just throw a GDExtension together for TIFFs.

[MarioLiebisch]

Just thinking about, maybe this would be an interesting option to have some automatic conversion function? As you say, imagemagick and other similar toolsets already support this. How about invoking it automatically upon import, just like Blender is used to import Blender projects? Stuff like this would allow the engine to utilize many different modern and legacy formats that are rarely used, but still used, without adding explizit support other than a extension list).

[Calinou]

Just thinking about, maybe this would be an interesting option to have some automatic conversion function? As you say, imagemagick and other similar toolsets already support this. How about invoking it automatically upon import, just like Blender is used to import Blender projects? Stuff like this would allow the engine to utilize many different modern and legacy formats that are rarely used, but still used, without adding explizit support other than a extension list).

Most people don't have ImageMagick installed, and it's considered niche functionality to most users. This also wouldn't work in the Android and web editors.

Blender import was added because it's often used in most 3D projects, and it's the software you actually use to create 3D models. (It's also not actually feasible to write a full-featured Blender importer without calling Blender, given how complex the format is.)

[fire]

It seems like https://en.wikipedia.org/wiki/GraphicsMagick is the successor to ImageMagick. Let us know if there's sufficient interest.

[fire]

@MarioLiebisch can you write a proposal for GraphicsMagick to openxr and GraphicsMagick to lossless webp inside of godot engine?

Similar to blender import.

[Calinou]

It seems like en.wikipedia.org/wiki/GraphicsMagick is the successor to ImageMagick. Let us know if there's sufficient interest.

The same issues as ImageMagick apply with GraphicsMagick, unfortunately. If anything is done with it, it's best implemented as an add-on.

[BurnoutBeat]

Still no work on Tiff Support?

[Calinou]

Still no work on Tiff Support?

To my knowledge, nobody is currently working on this. This could also be better suited as an extension, as TIFF is not something most projects will need and is subject to frequent security issues compared to other formats.

[false-fox]

plus support, TIFF support is useful when using geographic information system software for heightmaps and terrain, currently have to do a really weird workflow to get my TIFFs into raw or png

[fire]

For heightmaps and terrains openexr supports up to 32 bit floats and at least 16bit floating point.

[feefladder]

Just came here to add more to tiff support. Basically TIFFs are everywhere in GIS-land. Especially now with networking things, there is also something called Cloud-Optimized Geotiff, which makes use of internal tilings and HTTP range requests. I think it may be a bit of a niche use-case within the gaming industry, but I think it should work and not be too hard. I'm just struggling to understand a little bit how images work in Godot, so here's a bit of a breakdown. Also in GeoPackage (another GIS format, which is basically a SQLite database), if the image is stored as a floating-point raster, it is saved as tiff.

Going by this article, I think it should be a module? or a plugin? Basically desired functionality would be to have a load_tiff_from_buffer-type functionality

[feefladder]

Is a plugin an addon?

[Calinou]

Is a plugin an addon?

Yes, the term is essentially interchangeable.

[TokisanGames]

Recently I've had a few Terrain3D users who come from the GIS world ask about GeoTiff support (Tiff 6.0 + tags) and a workflow for converting to EXR for import. I've been promoting Terrain3D (and Godot) as a GIS visualization tool since its first release.

Occasionally I get 3D assets that come with tiff. In fact, just yesterday I was processing some rock packs that came with jpg, png, tga, and tif files.

[feefladder]

This is going somewhat off-topic, but the default open source raster enginge (GDAL) is not compiled with EXR support by default, so it is very well possible to export to EXR from tiff using GIS tools, but not so easily, since one has to compile a rather large program

[fire]

Tiff is a large specification to support so there is not that much of a will to have a second HDR format.

There's a png based hdr format that may have support in the 3d scene format gltf in the future but it's not TIFF.

[feefladder]

The specification has quite some overlap with exr:

• tiles or strips
• deflate compression
• overviews
• "ghost areas" for additional data additional things:
• LZW compression (used already by ThorSVG) differences:
• tags
• I couldn't really find how exr files know where each tile/overview is within the file, but they do support random tile order, so there should be a pointer/offset thing somewhwere in the spec.

I'm not suggesting to give full support LibTiff. There is also tinytiff, but that doesn't support compression or tiles. afaik, in gdextension you cannot link against those pre-existing libraries in the core? With limited (read-only) support, I think additional size would not be impacted that much, but rather legacy code and/or maintenance would be the problem?

[MarioLiebisch]

GDExtensions can do anything a regular program or library can do (because they're just that), unless you're in a very limited environments (such as Web Assembly). It would totally be possible to put LibTiff in a standalone GDExtension, although I never tried implementing a new format filter so far.

[feefladder]

Now it just happens to be that I'm targeting WebAssembly. I'm not super good at C++ programming, especially build systems, so I have not tried compiling LibTiff or TinyTiff to WebAssembly. I know that an extension could virtually re-create a big part of the engine. Specifically, it would be nice if DEFLATE and LZW compressions (on a byte stream) could be exposed. Then, writing a bare-bones decoder should not be that hard.

I made some notes for my use-case, here they are: What a rudimentary tiff reader should be able to do:

1. read the header
   • including:
     • BigTiff
   • excluding:
     • big-endian
     • non-interlaced (not RGBRGBRGB but RRR .. GGG .. BBB)
2. traverse internal file directories (IFDs)
3. read the IFD (geo) tags
   • for tiled:
     • extract single tiles
   • for striped:
     • extract the entire image data
4. when extracting, support the following compressions:
   • LZW
   • DEFLATE (Zip)

This would make reading GeoPackages and GeoTiff files possible. I get that it may be an extension. Is there any chance of decompression algorithms being (or already being) exposed?

I looked a bit at the TinyEXR implementation in the engine modules and it seems to be rather similar, but that would be a module, not an extension.

[Vertoletov]

TIF is very usefull to store alpha channel. If alpha value is 0 then photoshop discard pixels, but they still may contain useful information that can be used by a shader. It is commonly used for packed PBR information in 4 channels for example. And can't be achieved with png.

[Calinou]

If alpha value is 0 then photoshop discard pixels,

It should be possible to instruct Photoshop not to strip color data from fully transparent pixels in its save options dialog. GIMP no longer strips color data in this situation starting from 2.10.0.