Add support for KTX2

[UX3D-nopper]

see https://github.com/KhronosGroup/KTX-Software/tree/ktx2

[kopaka1822]

Hi, I'm currently rewriting the core engine to support easier unit testing and a console interface. I'll take a look at ktx2 files afterwards. It would also be helpful if you could provide some sample code how to load/export ktx2 files since I can't find it in the readme.

[UX3D-nopper]

Maybe a good start: https://github.com/ux3d/glTF-IBL-Sampler/blob/master/lib/source/ktxImage.cpp

[kopaka1822]

The link does not work for me. Maybe because it is a private project?

[UX3D-nopper]

Yes, you are right. Here is the public one: https://github.com/KhronosGroup/glTF-IBL-Sampler/blob/master/lib/source/ktxImage.cpp

[UX3D-nopper]

We also started to work on it, but did not have time to finish: https://github.com/ux3d/ImageViewer

[kopaka1822]

I figured out how to load ktx2 files with the ktx.h header. However I still have unresolved dependencies for the ktxTexture2_CreateFromNamedFile function. Do you know which of the ktx projects I have to build to resolve those dependencies? You can check out the ktx2 branch to try this yourself as well. See ktx_interface.h in DxImageLoader.

[UX3D-nopper]

@MarkCallow Can you please help on this?

[MarkCallow]

You need the libktx project from the ktx2 branch of the KTX-Software repo. See BUILDING.md for build instructions.

There are examples in the documentation but unfortunately until I move the ktx2 work to master I don't know how to make the Doxygen built documentation available via GitHub. In the meantime you can look at the loadtest programs. The simplest OpenGL {,ES} 3 one is DrawTexture.cpp. This also shows transcoding. There's an equivalent Vulkan load test as well.

Be aware that the transcode target format enums shown in the examples will be changing when the latest update from Basis is integrated. There'll be more of them and the names will change. That work is in progress.

[kopaka1822]

I added some experimental support for loading ktx2 files on the ktx2 branch. Most of the images from the KTX-Software test folder can be loaded. However, there are some issues with DXT-compressed formats and files that don't have a vkFormat specified. Let me know if you need support for specific file formats that don't work at the moment. If it works for the files you work with I can add an export for ktx2 as well

[MarkCallow]

However, there are some issues with DXT-compressed formats

Is this a problem in libktx or are you not using that?

and files that don't have a vkFormat specified

Is this referring to textures supercompressed with BasisU or something kind of file with a vkFormat?

[kopaka1822]

Is this a problem in libktx or are you not using that?

I am using libktx to load the ktx2 file. However, I convert it to RGBA8 using Compressonator because the image viewer only uses RGBA8 and RGBA32 internally to make some stuff easier. I don't have problems loading bc2 (DXT3) files from .ktx or .dds though. When I load "cubemap_yokohama_bc3_unorm.ktx2" the alpha channel is 255 for every 4th channel and 0 for the other 3/4 channels. If I overwrite all alpha values with 255, everything looks correct.

^ This is what it looks like zoomed in (the checkerboard means alpha = 0)

Is this referring to textures supercompressed with BasisU or something kind of file with a vkFormat?

Oh, your right, it's actually supercompressed with BasisU

[MarkCallow]

The may be a problem with vkFormat in cubemap_yokohama_bc3_unorm.ktx2. The original .ktx file has a glInternalformat of 0x83F3 which is GL_COMPRESSED_RGBA_S3TC_DXT5_EXT. DXT5 is now known as BC3.

In the conversion to .ktx2 vkFormat has been set to VK_FORMAT_BC2_UNORM_BLOCK. The difference between BC2 and BC3 is the way alpha is handled so the mislabelling may be the cause of the issue you are seeing. I will need to fix the conversion table and respin the .ktx2 file.

I do not see any issue with the texture in my test program. My program's reflection shader only uses rgb. The skybox shader passes all 4 components as its output color but there is no background to see through to, so possibly I just missed the problem. Also I have nothing to compare my result to.

Apologies for the problem.

Oh, your right, it's actually supercompressed with BasisU

What is "it" referring to? cubemap_yokohama_bc3_unorm.ktx2 is not BasisU compressed. I need to create an uncompressed version of the texture first and I haven't yet had time, though I do have the original images now.

The latest BasisU transcoder support in libktx lets you transcode to RGBA8.

[MarkCallow]

I have created PR #138 in KTX-Software which fixes the affected testimages. The conversion table had been fixed but the test images had not been respun.

[UX3D-wahlster]

Hi, the mipmap padding has been updated https://github.com/KhronosGroup/KTX-Software/pull/171 and some ktx files might not be compatible anymore. Could you please update the viewer to use the lastest version libktx?

Thank you for maintaining this viewer :)

[kopaka1822]

I updated the ktx submodule and the libktx binaries. Additionally, I added support for KTX_SUPERCOMPRESSION_BASIS files.

[MarkCallow]

All the ktx2 software is in KTX-Software master now. This also means the generated documentation is available at https://github.khronos.org/KTX-Software/. See https://github.khronos.org/KTX-Software/libktx/index.html#overview for a concise summary of how to use the library.

[kopaka1822]

Hello, I finally had some time to adjust my ktx2 loader to the latest version. The viewer is now able to load all ktx2 files from the ktx/tests/testimages directory. Do you need an export option for ktx2 files from within the viewer? If so, I can try to do that in the future.

[UX3D-nopper]

Great, looking forward to it. Will revisit to download the binary as soon as ready.

[kopaka1822]

I have a question for @MarkCallow What are the full requirements for ktxTexture2_CompressBasis() to work? The docs say that:

• the texture should not be supercompressed
• the texture format should not be in a block compressed format
• the texture dimension should be 2D

However, I tried to compress a small 3x3 image with VK_FORMAT_R32G32B32A32_SFLOAT and I got a KTX_INVALID_OPERATION as well. Is this limited to specific formats? Or does the image size also play a role? (For example: must be multiples of 4 or 8 in each dimension)

[MarkCallow]

There are 2 other conditions which I will add to the documentation. Sorry for the omission.

• @exception KTX_INVALID_OPERATION The texture image's format is a packed format (e.g. RGB565).
• @exception KTX_INVALID_OPERATION The texture image format's component size is not 8-bits.

This case is failing because the image format is 32-bit SFLOAT.

You can try supercompressing it with zstd (--zcmp option in toktx). If you do, I'd like to know how effective it is. I don't have many float textures available for testing.

[kopaka1822]

I started developing the export for ktx2 files and the export for non-basis-compressed should be finished. However, I still have some questions about the compression. First of all, I can compress the texture in either etc or astc format. In which cases should etc or astc be preferred?

Second thing is, I am not really sure how to properly import those textures again. I am not sure how to select the best format for the Transcode Basis function. Right now I use KTX_TTF_ASTC_4x4_RGBA for all cases. Another problem with transcoded texture is, that I don't know how to get the information about the original texture format before the transcoding (since vkFormat is zero before transcoding). I would like to display that information in the viewer if possible.

Last thing I noticed: when I compress an RG8 texture and import it afterwards, I get an RA8 texture.

I would appreciate if you can give me a few pointers @MarkCallow

[MarkCallow]

Please see the Developer and Artist Guides for your questions about when to choose UASTC vs ETC1S and for choosing transcode targets.

You cannot get information about the original VK_FORMAT before transcoding. It's not very useful. You can find out how many components were in the original image from the channel names in the DFD. This can be useful for choosing the transcode target. The other important information is the transfer function also available in the DFD.

Which version of the KTX-Software are you using? The latest version of toktx treats a 2-component import as luminance alpha (because that is what the PNG spec. says it is) so you get actually an RRRA texture. You can override this using the new --target_type option. If you specify RG and are encoding to UASTC you will get an RG01 UASTC texture. If encoding to ETC1S you get an RRRG ETC1S texture for better compression results. The transcoder can transcode this to a BC5 2 channel texture.

[MarkCallow]

so you get actually an RRRA texture

I was referring to UASTC and ETC1S encoding here. If you create a texture in an uncompressed format you indeed get an RA texture but it has swizzle metadata set that indicates you should apply an rrra swizzle for correct rendering. I did not want to nearly triple the size of the texture by doing the swizzle on input.

[kopaka1822]

The image viewer should now be able to export all supported .ktx2 texture formats. Formats that also support supercompression have a "quality" settings, which allows the user to select the quality of the supercompression, or disable supercompression: As you can see, they will be informed that quality below 100 results in compression. For exports to any SRGB format, the compression defaults to ETC1S w/ BasisLZ. For exports to UNORM formats, the compression defaults to UASTC w/ Zstandard.

As far as I could tell, those compressions only work on UNORM or SRBG file formats. So I disabled this setting for SNORM formats (and the other formats that do not fit the requirements as discussed in previous comments).

Currently there is no option to export SRGB formats with an UASTC compression. Some other specific options, like the "normalMap" param, are also not selectable.

Is there a need to add one of those options? Or does it sound good as it is? I need to do some testing before I will publish the next release. But I will inform you as soon as this is done.

[MarkCallow]

Sounds great. I'll take a look as soon as I have time.

"Quality" is the wrong term for supercompression (i.e. zstd) as it is lossless. The parameters basically determine how long compression takes vs the size of the result. It is fine for UASTC and ETC1S. The BasisLZ supercompression is tightly coupled with the ETC1S encoder so what you show in the dialog is fine for that case. For UASTC there are basically 3 knobs to twiddle: UASTC compression parameters, RDO parameters for conditioning the UASTC data for better LZ compression and the zstd parameters. The last is what I referred to above. If you ever add an option for applying zstd to existing data, that is when to avoid "quality".

UASTC is fully capable of handling sRGB. Users should be able to choose UASTC for sRGB inputs.

Is there a need to add one of those options?

Which options? If this includes sRGB/UASTC I've just answered. If this includes normalMap, since you don't have explicit settings for the RDO parameters and you are using RDO at some "quality setting" then yes you need to option so that RDO can be disabled.

[kopaka1822]

Okay, so my plan is the following:

For SRGB exports: By default ETC1S w/ BasisLZ is used. There will be an additional checkbox: "Use UASTC for compression", that will be disabled by default (because I think ETC gives much better results especially regarding the small file size).

For UNorm exports: There will be an additional checkbox: "Normal Map", that will disable the RDO.

The Code in the backend then looks like this:

ktxBasisParams params = {};
params.structSize = sizeof(params);
params.threadCount = std::thread::hardware_concurrency();
params.compressionLevel = KTX_ETC1S_DEFAULT_COMPRESSION_LEVEL;
params.qualityLevel = std::max((quality * 254) / 99 + 1, 1); // scale quality [0, 99] between [1, 255]
params.normalMap = KTX_FALSE;
if (!gli::is_srgb(format)) // only valid for linear textures
    params.normalMap = get_global_parameter_i("normalmap") ? KTX_TRUE : KTX_FALSE;

// select uastc for everything that is not color (here: for everyhing that is not SRGB)
// unless the "uastc srgb" flag is set => then use usastc as well
if(!gli::is_srgb(format) || get_global_parameter_i("uastc srgb"))
{
    params.uastc = KTX_TRUE;
    params.uastcFlags = KTX_PACK_UASTC_MAX_LEVEL; // maximum supported quality
    params.uastcRDO = params.normalMap ? KTX_FALSE : KTX_TRUE;
}

Does that sound good enough?

[j-cano]

First of all, thank you very much for this application. It's the only viewer I have been able to use to load KTX2, although I tried others that claim to support it and failed.

Is support for orientation (https://github.khronos.org/KTX-Specification/#_ktxorientation) planned? I am setting it up and down, but I see the same result, so I guess it's not there yet.

Thanks again.

[kopaka1822]

Hello, I thought that I did already implement the up and down support for ktx2. It would be easiest if you could attach the file, so that I can debug it. Support for left, right, inward, outward was not planned yet. Do you need that as well? If there are other issues with the ktx2 support, please report them. I rarely use that format.

[j-cano]

I'm sorry, I checked and rechecked and still the problem was in my test. Thank you for your fast answer that stopped me wasting more time and sorry again for wasting yours. I don't think I'll need other orientations and haven't found any other issues, but I'll report them if I do.

Thanks again, great app :)

[kopaka1822]

I will close this issue for now. Support for ktx2 should be stable. I tested against the files in the ktx repository. If anything does not work, feel free to reopen the issue

[j-cano]

Hi again,

I'm testing BC5 files now with normal maps. I'm using ImageViewer as a visualization tool. I created KTX2 files with betsy and Compressonator to check that I get similar results and that seems fine. I have tested BC5 with UNORM and SNORM, expecting to use SNORM because it seems more appropriate for normals, but viewing the files with ImageViewer doesn't feel right (I don't really know how it is supposed to be seen, with all those negative values, but it looks so broken that it feels impossible to know if it is right or wrong). Could you check this and confirm if this is how you are supposed to view these files? In the next image you can see the left file is the UNORM version and the right one the SNORM version:

I uploaded all the relevant files here (https://www.dropbox.com/s/15z30z56sdxjb76/test_bc5.zip?dl=0). Original png, results of converting to KTX2 with betsy and compressonator as UNORM and as SNORM.

Sorry in advance in case my test is wrong again.

Thanks.

[kopaka1822]

Hello, I would first like to answer your actual question and then point out some potential problems.

About viewing signed data: By default, the viewer will show absolute pixels values (so, -0.4 will be displayed like positive 0.4). This can be changed in the menu bar via View->DisplayAbsoluteValues.

After changing this setting, you could change the image equation from I0 to -I0 if you want to see everything that has a negative sign. (See https://github.com/kopaka1822/ImageViewer/blob/master/Docs/getting_started.md#image-equations for details, I recently wrote a guide for getting started ;) )

However, I can't help you further because I don't know in what kind of format your normals are after they have been exported. My problem is that BC5 is a two-channel format => as you can see, only the red and green channels are used. This means the original XYZ representation was changed to something different (maybe like theta phi or some other mapping?). If you let me know how to interpret these two channels I can help you further.

Last I would like to point out how you can view the signed normal data of your original.png. Take a look at the image below:

You can see in the first red box (after hovering over the imported image with the mouse), that you png file is encoded as SRGB. However, normal maps should be interpreted as linear. You can reinterpret SRGB data as unorm with the SrgbAsUnorm() function. As a last step, you need to multiply the texel with 2 and subtract 1 to get from the uniform [0, 1] range to [-1, 1] range (This is the default encoding used for normal maps). If you have never heard of SRGB, see https://learnopengl.com/Advanced-Lighting/Gamma-Correction

You can also verify, that this normal map is valid: By using the length() function in the bottom red box I visualized the length of each normal. As expected, all lengths are very close to 1.0, so the original.png appears to be fine. In the picture above, you see the normal map on the left side (with absolute signs) and the lengths of each normal on the right side (completely white here).

I hope I could this helps a little. Please let me know how the BC5 normal maps need to be interpreted (Its probably somewhere in the betsy/compressonator docs?)

[MarkCallow]

My problem is that BC5 is a two-channel format => as you can see, only the red and green channels are used. This means the original XYZ representation was changed to something different (maybe like theta phi or some other mapping?). If you let me know how to interpret these two channels I can help you further.

The, er, normal way of handling this is normalize the input normals, to get unit normals, and then discard the z component. A shader can recover the z component by using the equations:

nml.xy = texture(...).ga;              // Load in [0,1] *
nml.xy = nml.xy * 2.0 - 1.0;           // Unpack to [-1,1]
nml.z = sqrt(1 - dot(nml.xy, nml.xy)); // Compute Z

// * When using 4-component block compression schemes X & Y are stored in the green
// and alpha components of the compressed texture because there is no correlation between
// these components in the compression scheme

[kopaka1822]

Thank you for the help Mark Callow.

I have confirmed that your unorm files are almost identical to the original normal map, after proper conversion. Let me explain the procedure: As I explained in the previous comment, you can extract the normals form your normal map .png file with srgbAsUnorm(I0)*2-1 (if I0 = original.png). The formula of mark callow can also be expressed with: RGB( r(I1*2-1), g(I1*2-1), sqrt(max(1 - dot(I1*2-RGB(1, 1, 0), I1*2-RGB(1, 1, 0)), 0))) (with I0 = ...bc5unorm.ktx2). The last formula is really long, so let me explain. The RGB() function takes three parameters and uses them for the red, green, and blue channels respectively. For red I use r(I1*2-1) which reads the red channel after doing the normalization from [0,1] to [-1,1]. For green I use g(I1*2-1) which does the same with the green channel. For blue I use the normalization from the above comment: sqrt(max(1 - dot(I1*2-RGB(1, 1, 0), I1*2-RGB(1, 1, 0)), 0))). However, I added the max function in case the value inside the square root becomes smaller than 0 (which happens for ~3 texels in your texture). Note that I use I1*2-RGB(1, 1, 0) here to make sure, that the blue component remains at zero. This is important because the dot function usesrgb values.

However, the snorm files appear to be broken. I would have expected that they give the same results as unorm but without the normalization (RGB( r(I1), g(I1), sqrt(max(1 - dot(I1, I1), 0)))). They have a lot of broken texels if you see below:

If you have any questions @j-cano feel free to ask. I was always wondering if I should write a guide for normal maps since they can be confusing at first.

[j-cano]

Thank you both for your help.

I have been able to load these KTX files in my OpenGL app and I can confirm they are not broken, not even the SNORM files. In fact, they both look almost the same when trying to paint the same (both in the 0 - 1 range after recalculating z), which is what I expected since the change is about how the file is stored and the data is logically the same. However, it seems very important to choose the right format when uploading the texture. I can't upload the SNORM file as GL_COMPRESSED_RG_RGTC2 and then fix it in the shader, I have to upload it as GL_COMPRESSED_SIGNED_RG_RGTC2. Reading from https://learn.microsoft.com/en-us/windows/win32/direct3d10/d3d10-graphics-programming-guide-resources-block-compression#bc5 I guess some information is lost forever in that signed/unsigned math if done incorrectly. Maybe something similar is happening here?

@MarkCallow I think I understand what you are saying about using the green and alpha channels, but isn't BC5 suppossed to be red and green? Should I use the KTXswizzle in this case to remap them to green and alpha? Or this is something that should happen in the compression process? Is this documented somewhere? In the KTX specification (I know you wrote that, but in case anyone else needs it is in https://github.khronos.org/KTX-Specification/) it says that the VK_FORMAT_BC5 maps to GLCOMPRESSED*_REDGREEN\. So, I would expect to read the data in the red and green channels, right? Microsoft also talks about red and green, so I am a bit confused about this.

Thanks again.

[MarkCallow]

@j-cano Sorry for the confusion I caused. Green-alpha is what is usually used in a 4-component block compression scheme. When you have an explicit 2-component scheme like BC5 you use whatever the scheme has labelled those components, so R & G for BC5.

If using Basis Universal, the normals' XY components will be stored in G & A. However if you transcode to BC5 they will be in R & G in the resulting BC5 texture.

[j-cano]

Thanks, Mark.

@kopaka1822 I have been doing some debugging with the code. I think that in compress_interface.cpp, the return format for FORMAT_RG_ATI2N_SNORM_BLOCK16 should be CMP_FORMAT_BC5_S instead of CMP_FORMAT_ATI2N_XY. That way, the compressonator CCodec_ATI2N_S is used instead of CCodec_ATI2N and values are decoded as signed, where previously where decoded as unsigned. There is still something weird with the result, the red channel is shown as 1, green and blue have values between -1 and 1, which I expected for green, but not for blue. I tried some swizzle using the equations, but didn't found anything. As Mark said, I would expect data to be only in red and blue. I hope this helps. I will tell you if I find anything else.

[j-cano]

I tested some more. I think in compress_interface.cpp, get_cmp_format should return CMP_FORMAT_RGBA_8888_S for FORMAT_RGBA8_SNORM_PACK8. I also had to comment the overwriteAlpha for FORMAT_RG_ATI2N_SNORM_BLOCK16 because if not commented there is an assertion that fails. I don't think it's necessary because compressonator already sets alpha to 127 when applying the change of my previous comment. With all that changes and applying the simple equation "rgb(red(I0) 0.5 + 0.5, green(I0) 0.5 + 0.5, 0)" the image shown in ImageViewer is the same as with UNORM doing nothing. I think that's what should happen.

What do you think?

[kopaka1822]

Hello, thank you for your debugging effort. While fixing the issue I also discovered the same problem with the ATI1 format, so I fixed that as well in the latest commit

[j-cano]

Great! I don't think I will be trying more formats soon and everything seems fine, so that's all for now.

Thanks again!