Feedback

[iMrShadow]

Hi!

I finally had the time to test out the bindings. Unfortunately, I got into an issue. I want to assign the original byte array texture to a ktx2 one. Here is what I tried to do:

byte[] d3dtxTextureDataArray = textureData.SelectMany(b => b).ToArray();

fixed (byte* ptr = d3dtxTextureDataArray)
{
texture.PData = ptr;
texture.DataSize = (uint)d3dtxTextureDataArray.Length;
}

//...

fixed (Texture* ptr = &texture)
{
Ktx2.WriteToNamedFile(ptr, newKTX2Path);
}

I am almost certain I did something wrong, I get an exception when trying to save to a file (presumingly it doesn't allocate memory correctly): An unhandled exception of type 'System.AccessViolationException' occurred in Ktx2.NET.dll: 'Attempted to read or write protected memory. This is often an indication that other memory is corrupt.'

The file is created on the system, but there is no data inside it. I tried looking up the in the Khronos documentation, but in their samples they use ktxTexture_SetImageFromMemory.

Other than that so far it has been clean. I was also wondering if IsSRGB and ktx2ktx2 can be added as well (I could use it to transform to ktx1 and then transcode to rgba8).

Thanks!

[BoyBaykiller]

Thanks for the feedback!

I did just try out Ktx2.WriteToNamedFile myself (in combination with Ktx2.CreateFromNamedFile) and it worked, so I think there is something wrong with your usage.

Two things I notice:

1. You are letting PData point to managed memory byte[]. This is fine as long as the array is fixed, but after that the GC can move/delete it in and the pointer will become invalid. Consider using unmanaged memory, with the NativeMemory class.
2. Why is texture getting fixed? It should be of type Ktx2.Texture* which you got from either Ktx2.CreateFromNamedFile or Ktx2.CreateFromMemory. No fixed required.

And be sure to check the error code, although I know this is useless in this case since it crashes.

---

ktx2ktx2 seems to be some command line utility. This is out of scope. You can install it from here and then call the exe with Process.Start() from within your project. IsSRGB I might add in the future.

[iMrShadow]

Hi! Thanks for the response. ktx2ktx2 being a CLU is a shame, since it would have been useful for in-memory conversions. I am trying to display as many surface formats as possible in my app from KTX, KTX2 and DDS files. However, I need to convert them to RGBA32 first, because Avalonia's Image only supports RGBA32. I am not sure how else I can get it working without going super deep.

You are letting PData point to managed memory byte[]. This is fine as long as the array is fixed, but after that the GC can move/delete it in and the pointer will become invalid. Consider using unmanaged memory, with the NativeMemory class.

Why is texture getting fixed? It should be of type Ktx2.Texture* which you got from either Ktx2.CreateFromNamedFile or Ktx2.CreateFromMemory. No fixed required.

I want to create entirely new KTX2 textures from the data I managed to parse from the proprietary textures. Here is the full Master class for full context with some updated code.

I am planning to add more functions using the ones you added.

using System;
using System.Collections.Generic;
using System.IO;
using D3DTX_Converter.DirectX;
using System.Linq;
using static Ktx.Ktx2;
using Ktx;
using System.Runtime.InteropServices;

namespace D3DTX_Converter.Main
{
    /// <summary>
    /// Main class for managing D3DTX files and converting them to KTX2.
    /// </summary>
    public class KTX2_Master
    {
        Texture texture;

        /// <summary>
        /// Create a KTX2 file from a D3DTX.
        /// </summary>
        /// <param name="d3dtx">The D3DTX data that will be used.</param>
        unsafe public KTX2_Master(D3DTX_Master d3dtx)
        {
            if (d3dtx.IsLegacyD3DTX())
            {
                throw new NotImplementedException("KTX2 does not support legacy versions.");
            }

            // Initialize the KTX2 Header
            texture = new()
            {
                BaseHeight = (uint)d3dtx.GetHeight(),
                BaseWidth = (uint)d3dtx.GetWidth(),
                BaseDepth = (uint)d3dtx.GetDepth(),
                VkFormat = KTX2_HELPER.GetVkFormatFromTelltaleSurfaceFormat(d3dtx.GetCompressionType(), d3dtx.GetSurfaceGamma(), d3dtx.GetAlpha()),
                NumLevels = (uint)d3dtx.GetMipMapCount(),
                NumLayers = (uint)d3dtx.GetArraySize(),
                NumFaces = (uint)(d3dtx.IsCubeTexture() ? 6 : 1),
                IsArray = d3dtx.IsArrayTexture(),
                IsCubemap = d3dtx.IsCubeTexture(),
                IsCompressed = d3dtx.IsTextureCompressed(),
                NumDimensions = (uint)(d3dtx.IsVolumeTexture() ? 3 : 2)
            };

            List<byte[]> textureData = [];

            // Get all pixel data from the D3DTX
            var d3dtxTextureData = d3dtx.GetPixelData();

            int divideBy = 1;

            // Get the pixel data by mipmap levels and unswizzle depending on the Platform
            for (int i = (int)(texture.NumLevels - 1); i >= 0; i--)
            {
                textureData.Add(d3dtx.GetPixelDataByMipmapIndex(i, d3dtx.GetCompressionType(), (int)d3dtx.GetWidth() / divideBy, (int)d3dtx.GetHeight() / divideBy, d3dtx.GetPlatformType()));
                divideBy *= 2;
            }

            // Put all pixel data into a single array
            byte[] d3dtxTextureDataArray = textureData.SelectMany(b => b).ToArray();

            // Attempt to put the pixel data into the texture
            IntPtr ptr = (nint)NativeMemory.Alloc((nuint)d3dtxTextureDataArray.Length);
            Marshal.Copy(d3dtxTextureDataArray, 0, ptr, d3dtxTextureDataArray.Length);

            texture.PData = (byte*)ptr;
            texture.DataSize = (uint)d3dtxTextureDataArray.Length;
        }

        /// <summary>
        /// Writes a D3DTX into a KTX2 file on the disk.
        /// </summary>
        /// <param name="d3dtx"></param>
        /// <param name="destinationDirectory"></param>
        unsafe public void WriteD3DTXAsKTX2(D3DTX_Master d3dtx, string destinationDirectory)
        {
            string d3dtxFilePath = d3dtx.filePath;
            string fileName = Path.GetFileNameWithoutExtension(d3dtxFilePath);

            // Create a new path for the KTX2 file
            string newKTX2Path = destinationDirectory + Path.DirectorySeparatorChar + fileName +
                                                       Main_Shared.ktx2Extension;

            // Attempt to save the new KTX2 file
            fixed (Texture* ptr = &texture)
            {
                Ktx2.WriteToNamedFile(ptr, newKTX2Path);
            }
        }

        // I have to somehow free the byte memory. The intended use is Texture*, but that wouldn't let me change the parameters. 
        unsafe ~KTX2_Master()
        {
            fixed (Texture* ptr = &texture)
            {
                Ktx2.Destroy(ptr);
            }
        }
    }
}

Thanks!

[BoyBaykiller]

Ok I see. I only have a limited understanding of KTX and I am unfamiliar with .d3dtx. But the way you are manually assigning the KTX2-Texture struct is obviously extremly dangerous and requires intricate knowledge of not only the specification but also the current implementation details. And I don't even expose all fields (there are some private fields for the library), so its doomed to fail. Ideally you'd only use the public KTX2 functions.

For example I found ktxTexture2_Create which creates a new empty KTX2-Texture given this ktxTextureCreateInfo struct. And from there on you could use functions like the ktxTexture2_SetImageFromMemory you mentioned to actually set the data.

[iMrShadow]

It would be nice to have those functions. D3DTX is Telltale's format for textures, which I successfully parse to DDS, however they also use OpenGL and Vulkan formats, which DDS doesn't support.

However, some good news - the way they store their data is the same as KTX2 (by mipmap levels, smallest to largest), so I don't even need to modify how the regions are ordered.

I just need a way to store that data in a readable container as you saw in the code segment I sent.

Thanks

[BoyBaykiller]

however they also use OpenGL and Vulkan formats, which DDS doesn't support.

Hm so DDS stores data in one of the GL formats

• GL_COMPRESSED_RGBA_S3TC_DXT1_EXT
• GL_COMPRESSED_RGBA_S3TC_DXT3_EXT
• GL_COMPRESSED_RGBA_S3TC_DXT5_EXT

And you are saying "D3DTX", the format used by "Telltale", can contain data for other GL formats (unspported by DDS) such as GL_COMPRESSED_RGBA_BPTC_UNORM. Do I understand correctly? So you can't work with all the D3DTX textures of the game?

[iMrShadow]

Something like that, yeah.

Telltale use their own cross-platform (and annoying) game engine. However, it was never published to the public.

These are all formats which their engine supports (they were found after decompiling their games). The first ~30-40 formats are supported by DDS (S3TC and uncompressed formats), which are found on PC, XBOX, and PS3/4 platforms. However, platforms such as Android or iOS use PVRTC (presumably version 1), ETC1, ETC2, ATC and ASTC, formats that are supported only by KTX and KTX2

[BoyBaykiller]

Alright. So should I add these?

• ktxTexture2_Create
• ktxTexture2_SetImageFromMemory

You tell me what you need.

[iMrShadow]

Here are all the things that can be added:

• ktxTexture2_Create with its createInfo struct
• ktxTexture2_SetImageFromMemory
• ktxTexture2_CreateCopy

Not super important but nice QoL:

• isSrgb
• GetImage (this is a combination of some existing functions)

Some quick questions:

• I got mistaken by TranscodeBasis, thinking it can transcode the texture from one format to another, but I wasn't familiar with .basis. Is there a general uncompress or transcode function? Example usage: we have a BC5, ETC2 and R4G4B4A4 textures. We want to transcode/change their formats to RGBA32. Unless I am mistaken again and it does work for all formats...
• Is it possible to determine if a texture is ETC1 compressed? It's not a Vulkan format (it's set to Unknown), but KTX2 supports it. Is it the determined by the supercompressScheme?

Thanks

[BoyBaykiller]

Is there a general uncompress function

Not sure, I don't think so. If you use a graphics API like OpenGL the uncompression is automatically done by the hardware when you sample a texture. Btw this how is determined wether a texture needs to be transcoded first.

Is it possible to determine if a texture is ETC1 compressed?

Yes, you can check if Ktx2.Helper.GetDataFormatDescriptor(ktxTexture).Model == 160. 160 because thats the value for ETC1 (I will add the enum values).

I'll try to add the functions the next days.

[iMrShadow]

Thanks

So there isn't anything similar to DirectXTex's function? (I forgot to reference what I was talking about, sorry about that).

Edit: there's this and this, but they write to a file, while I need it in-memory...hmm

There's also this - the encode command, but as I said - memory only, I don't want to write it on the disk

[BoyBaykiller]

So there isn't anything similar to DirectXTex's function?

I don't know if there is a function like that in KTX-Software. If you find a way to decompress the KTX image into raw pixels you can use something like stb_image_write.h to decoded it into a png/jpg in memory.

[BoyBaykiller]

@iMrShadow I've added the three functions and did some other changes in branch "stuff-for-issue-2". Here is the usage https://github.com/BoyBaykiller/Ktx2.NET/blob/stuff-for-issue-2/tempTest.cs

To try out my changes clone this repo and checkout branch "stuff-for-issue-2". Now you have two options:

1. Building this project will generate a .nupkg file. In your project add a local nuget source (under "Manage Nuget Packages" -> "Settings") to the folder containg this .nupkg file.
2. Building this project will generate a .dll file. In your project add this .dll as a assembly. However you'll need to make sure the native ktx.dll is found. You can hardcoding the path here.

Once you got everything working I will publish it to nuget!

[iMrShadow]

Hi! Thanks for the additions. I will make sure to try them out next week and report if everything's fine