kramv crashes opening most .ktx2 files

[MarkCallow]

I've tried several .ktx2 files from my test corpus. kramv crashed opening all of them except for a uastc/zstd compressed cubemap which it reported being unable to open (fair enough you don't claim to support uastc/zstd yet) and an etc2_rgba8 file which it opened successfully. One case which points to an issue with your KTX v2 handling is that when opening a BC2-compressed .ktx file kramv said it was unable to open it. When opening the exactly equivalent .ktx2 file, it crashed.

I will remove kramv from our Tools Guide until these issues are fixed.

Do you plan to support BasisU compressed .ktx2 files, that is ETC1S/BasisLZ and UASTC/zstd compressed files. Do you plan to support zstd compression on other texture formats where it makes sense, e.g, ASTC?

[alecazam]

Sorry, just saw this post. kramv is meant to load images compressed by kram. I don't support BC2 because it's dead format. I'm trying to direct users to better formats, not all formats. It offers no advantages that BC1/3/5/7 don't already handle. I also don't handle BC6 or ETC2RGBA1 files. This is all outlined on the home page.

I don't support UASTC or any basis content yet, since I don't pull in any Basis libraries. The python scripts can generate UASTC/Basis files though for size and timing comparisons. Currently, I use ktx2ktx2 and ktx2sc to convert, and I'm currently converting krams KTX + ETC2/BC/ASTC into ktx2 + zstd textures that way. These should all open in kramv which is the viewer app. kramv can open PNG, KTX, KTX2, and KTX2+zstd files. It can also cycle through images in a .zip archive of ktx or ktx2 files.

I am not using Basis since it limits block size to 4x4, and the sizes aren't any smaller for UASTC as for using ASTC currently. I found quality issues and encoding timing to be too slow. That was 1 year ago, so the quality and timing are getting faster. BasisLZ will never be enough quality with 5551 ETC1 endpoints. My company does all texture encodes locally on each persons machines, so fast encoding is important. Also looks like Rich is considering switching Basis to Rust, and I need to keep all this C++.

[alecazam]

If you look at encoders, BC7enc doesn't support BC2 and ATE doesn't support it either. The only encoder in kram that handles BC2 is Squish. I just have the format blocked from being specified by the CLI. kramv shouldn't crash, but maybe it needs to call the isFormatSupported call.

[alecazam]

I will remove kramv from our Tools Guide until these issues are fixed.

Seems a little harsh, since kramv can load KTX1 files perfectly. I only recently added the ktx2 load support.

[alecazam]

The one idea was to incorporate libktx to handle ktx2 decode/encoding which kram doesn't yet do. I'm not quite sure how much code size that will add to the current build, and complexity of being tied to a submodule. I need to build all this with Github Actions too, but I think libktx is built with TravisCI.

[alecazam]

The crashes were from an assert, that I had return false to fail the load with an error log message. None of the failures in libkram go out to a posted gui dialog. But at least it won't crash. I added early returns on unsupported texture formats to avoid this. The format table has a specific list of formats that are supported.

I was able to open quite a few of the images in libktx's testimages folder. See below for more detail.

[alecazam]

Okay, I made kram/kramv loading fail early on unsupported formats. It was able to open, decode, and encode all advertised formats. There's also a bc2 labeled texture in the corpus that is really BC3 cubemap_yokohama_bc2_srgb.ktx. I ran through all of them, and there are no more crashes, and the loader code earlies-out on unsupported formats.

I only handle 1/2/4 channels currently. S/RGB8 which many in your text corpus are currently unsupported. Dealing with the dword row alignment vs. non-dword in ktx2 wasn't something I wanted to deal with as I was writing all this. The same goes for mips of explicit 1/2 channel textures if those are specified. Internally all textures are stored as 4 channel 8/16f/32f with/without mips, and then they encode to the BC/ETC/ASTC formats. The conversion from RGB -> RGBA isn't tough, but it would be input only and never an output format.

I don't handle R10G100B10 which is another. There are ton of these explicit formats, and kram is not a general encoder. It takes a set of RGBA8u unmultiplied R/LA/RGB/RGBA PNG and R/RG/RGBA 8/16f/32f KTX/KTX2 source files, and can encode them back as KTX files for now. I don't handle UASTC or BasisLZ compression, so those are skipped too.

The idea is to use KTX2+zstd with Photoshop instead of PNG for explicit R/RG/RGBA textures, and then feed those to the encoder to get to KTX2 + BC/ASTC/ETC2 + zstd textures that are needed for shipping games. It can encode/decode and process all KTX texture types and many formats, and so that's a lot of work in and of itself.

[alecazam]

Those RGB 8/16F/32F formats don't exist in Metal, so I that's why I didn't support those. There's no exposed MTLPixelFormat for those.

[alecazam]

Okay, there's crude RGB 8/16F/32F support of SUPORT_RGB. It loads one mip from the KTX file if it's type 2D. That's all the more support I have. There are still row alignment issues not handled, but it opens many of the rgb files in your folder. Did you try opening any of your testimages in Preview, since many of these KTX don't open there either? That was the whole reason I wrote kram/kramv. There is also PVRTexToolGUI, but it has it's own issues.

[MarkCallow]

Just to be clear, I'm not asking you to support all formats. What I expect from the viewer is that it will tell you if a format is unsupported and it won't crash. I only mentioned the BC2 format file because with the ktx v1 file, kramv had my desired behavior, as just described, but with the ktx v2 version it crashed indicating issues with the ktx v2 container handling.

The Tools Guide's focus is ktx2 tools hence removing kramv.

isFormatSupported() is for only for transcode target formats. It is necessary as there are #if macros that enable omission of formats from the built libktx.

Yes, libktx is currently built with Travis CI but we want to switch to GH Actions. We'd welcome a contribution to switch the entire build. We're about to take baby steps in learning GH Actions by using it for the upcoming Android build.

The basisu encoders have recently been greatly improved in both quality and performance. The ETC1S encoder is 3x faster with no quality loss and 4.5x faster with a slight quality loss. The improved encoders were in KTX-Software 4.0.0-beta8 released on March 25th.

the sizes aren't any smaller for UASTC as for using ASTC

True. But on the other hand you only need 1 texture. You can losslessly transcode to ASTC and near losslessly transcode to BC7. You can transcode to other BC formats and ETC2 subject to the limitations of those formats. No need to keep 3 textures around. See our Developer Guide for extensive information about appropriate uses for each BasisU format and choosing appropriate transcode targets.

There's also a bc2 labeled texture in the corpus that is really BC3 cubemap_yokohama_bc2_srgb.ktx

I've removed that file. Not because of the mislabeling but because the coordinate system of its faces has been, er, butchered from the original face set and is not compliant with the tightened requirements of ktx v2.

R/RG/RGBA 8/16f/32f KTX/KTX2 source files

What do you use as the input format for these?

There are still row alignment issues not handled

Is this in the ktx v1 or v2 support?

I'll try the updated kramv tomorrow. No time today.

I've only tried a few of the .ktx files in Preview. Mostly it seems to support the same subset as Finder so I usually just look at the preview in Finder.

[MarkCallow]

You may be interest in the Switch-case generator we've just added to the KTX-Specification repo. It produces 5 files with simple C-like case-return statements mapping from VkFormat enums to, among others, Metal enums.

[alecazam]

The Tools Guide's focus is ktx2 tools hence removing kramv.

kramv supports ktx and ktx2 with or without zstd.

isFormatSupported() is for only for transcode target formats. It is necessary as there are #if macros that enable omission of formats from the built libktx.

I have an internal call for that in libkram. And a table of supported formats and properties. All I did was test against undefined on that table.

True. But on the other hand you only need 1 texture.

I've been down the Basis path a few times. I wouldn't have written kram if the ktx tools could encode to ETC/ASTC/BC, but these only seem to have gone down the UASTC route. I'm sure I'll need to support that for glTF2 files that might use it. But UASTC has several drawbacks as well. Since the ktx tools didn't support ETC2/BC/KRAM, that's why I had to use kram, and then use ktx2ktx2 for KTX2, and then ktx2sc for zstd encoding. I have everything in libkram to skip using ktx2ktx2 and ktx2sc, but need to finish ktx2 writing.

1. It needs memory to decompress the data, then more memory to transpile the data to a format that the driver can accept. This doesn't occur with ETC2/ASTC/BC that can decode directly from zstd mips to a staging texture. When one needs to load 100 images in a hurry, needing 2x the memory for that isn't ideal.

2. The encode times especially with RDO are only usable with incremental server-based content builds. At some companies, the content is built locally on every persons machine. Taking an hour to wipe and rebuild textures isn't something my company can afford. I've been working with Pete Bishop at ARM to fixup astc-encoder. When I started a year ago, a full texture build took 45 minutes and now it's around 1 minute for our full corpus of 500 textures. That's encoding to BC/ASTC/ETC for all 500. That 30s is a good case for UASTC+RDO encoding a single texture.

3. There are always things lost in a transcode (f.e. ASTC block size). I used Rich's BC7enc and it's fantastic, and I'm sure once Binomial's efforts are put into UASTC it will eclipse other encoders out there. Just not in its current form, but it is one way to solve the 1:many problem on Android. In our case, we just ship ETC2 and the problem is solved. I even wrote an optimized version of ETC2comp for libkram. ETC2 is universal across iOS/Android devices and has direct hardware support. ASTC block sizes above 6x6 are really not good at fitting data so UASTC 4x4 works okay for many. We just need textures on GPU to be even smaller, and disk size is less critical.

What do you use as the input format for these?

Those are my source formats. PNG, KTX, and KTX2 source files and KTX/KTX2 output. I might add PNG output for non-mip too. The idea is to start the round trip of KTX2 lossless in and KTX2 losless/lossy out. Then we can drop PNG and having to deal with unmultiplied alpha, flattened mips/cubes/atlases, file name munging, etc. TIFF is the closest to KTX/KTX2 as far as a source format, but again it doesn't have the gpu-specific settings that KTX/KTX2 hold so well.

Is this in the ktx v1 or v2 support?

Both in KTX and KTX2. All my textures are mostly pow2, and I handle non-pow2 png ad non-pow2 mips off PNG. It's just if sourcing from KTX files in libram that there is an issue due to the DWORD alignment. I've noted other KTX systems have to switch to a row-by-row read/write if they detect a 1/2/3 byte format that can be unaligned, and I need to do the same. Internally, all mip data in kram is tightly packed with no padding more like KTX2.

• Remaining issues I see in the KTX2 format.

Can I write a KTX2 file with a dfdByteLength = 0. That's the most complex structure of the KTX2 header, and my writer could skip that initially. The format already details a lot, and I use props for the rest. It's not like I don't have your sourcecode, just only have limited time to work on kram.

Props can sometimes be binary and sometimes ascii (even in KTX). In my loader, I assume all ascii text props and print them out. I shouldn't have to consult some ratified list of KTX props, but it would help to know what the prop payload type is. I think that KTXorientation was the only one in KTX, and there have been very few added to KTX2. Fortunately, ktx2ktx2 preserves my KTX props when going to KTX2.

glTF for example used json instead of a name-value pair blob. That was nice since it enforced that every reader could display and manipulate the data even if they didn't comprehend it all. I've been tempted to embed a single prop of json like that rather than multiple props.

No thumbnails or Apple Preview on KTX2. This is a pain for artist.

No Photoshop editing/saving of KTX or KTX2 format (at least on macOS). The Nvidia plugin is proprietary code, and goes way overboard on what it does. I want only lossless KTX2 + zstd formats (no BC/ASTC/ETC2) only RGBA8/16F/32 so that simplifies the save/load UI. This was the other biggie that I was working on.

[MarkCallow]

Can I write a KTX2 file with a dfdByteLength = 0.

You could but it would fail validation and it would also not be loadable by libktx which uses the DFD information to determine the format's block size and from that things such as image sizes, level sizes and the mip-level alignment requirement.

[alecazam]

Those values are already in my table, and everything is driven off the VKFormat in KTX2 anyways. But I guess premul and other state are specified in the DFD. Just would be bad to have a DFD that has redundant conflicting data from the format. I'll try to add the KTX2 write + DFD + ZSTD compression in my next update.

I sincerely hope that KTX2 doesn't solely wrap Basis UASTC/ETC1S just because the ktx tools only support that single encoder. Otherwise, I would just use a basis file that has more controls for encode and RDO.

[alecazam]

If mips weren't reversed to smallest first in KTX2, then could compress and in-place mip to a file. As it is now, the offsets for the largest mips aren't known until the smallest mips are generated. So all compressed data must be held in memory when building the mips. In my case, I'm just holding all uncompressed mip data in memory, and then compressing in another pass. But that can be a lot at large mip sizes.

[alecazam]

Okay, kram now writes KTX2 with DFD and zstd/zlib compression. kram and kramv provide verbose info on KTX2 files. And kramv now reports why it failed loading ktx2 files with USASTC or ETC1S BasisLZ compression. I may pull in the single-file basis transcoder for this, but not urgent on my list.

There are two of your test ktx files that fail from a mipmap mismatch, but I don't handle the 4-byte row alignment in ktx files. All of my test images load, and many of yours do. So many of your test ktx/ktx2 images are RGB8 no alpha. I can import those now, but prefer to stick to PNG for rgb data and only write out RGBA or block compressed textures. It was a lot of work to get KTX2 in and the code is still fresh. I haven't run my ktx2 files through the ktx file format checker yet.

I also didn't find any docs on the FSEL bits in the DFD. I gathered that F-float, and S-signed, but E-extended? and L-uminance? No idea there, and no examples either that I could see. The DFD really seems redundant with VKFormat and should be optional in my opinion. The premul state also isn't clear about whether premul was done to srgb or linear rgb data. I found Photosohop does it srgb a, and gimp/kram do premul against the linear a then convert to srgb.

kramv now has some ui buttons and can do some pretty great triage on ktx and ktx2 files. The debug modes should be especially handy. bc7enc encoder is setting constant block alpha incorrectly to 254. Also etc2comp is setting color rgb on some grayscale blocks. These all seem to be off by 1 errors, and the debug modes flag these pixels. I'll have to delve into the encoders next. I'm going to close this, since I think I met your requirements for a ktx2, but let me know if you need more.

[MarkCallow]

Sounds great. I'm overwhelmed with work on KTX-Software right now but I'll take a look as soon as I can. I look forward to it.

also didn't find any docs on the FSEL bits in the DFD.

Bureaucratic issues with Khronos have delayed release of the latest Data Formats spec. However the one that has been released has a description of the F, E & L bits. The L is linear. It indicates the sample is linear, i.e. ignores the Transfer Function setting. This is typically used for the alpha channel.

The DFD really seems redundant with VKFormat

Only if you create a lot of tables indexed by the VkFormat enum to provide the information to your program.

The debug modes for the encoders sound really useful.