Extract a specific mipmap from a texture instead of fullchain if texture meta supports it.

[kyapp69]

In our application we sometimes have the need to extract a particular mipmap level and create a texture from it instead of loading the full mipmap chain. The reason for this is to be able to control the amount of graphics memory used at the expense of visual quality. One way we can do that is to calculate the bytes needed per mipmap and basically offset from the start, but we need a way to calculate that based on the image's bpp. If it's possible in the current codebase, can you give me some pointers?

[atteneder]

Hi @kyapp69 ,

Would you need a single mipmap only or a chain starting at a higher than 0 level? I'd say both can be useful, depending on the use-case.

The texture mipmaps are transferred from libktx (C/C++) to managed C# via ktx_copy_data_levels_reverted, which copies all mipmaps (and re-orders them).

We'd have to either alter it (to take a "starting level" parameter) and/or create a variant that extracts only one level.

Once that native part is done, the C# needs to be implemented to close the gap.

Let me know if you're going to tackle it and if you need more help.

[kyapp69]

We are only currently using basis files as a start vs the .ktx container. I can see both being useful as for the full mip chain case it can be applied for the different platform use cases ie.. mobile footprint all the way up to console footprint use via a single basis asset. In fact our assets are created that way, we encode at the highest resolution for the forseeable future and pick the resolution/mip chain during runtime for the particular platform. That way we only need to author it once. For the case of the single mipmap only, it's used in cases ie UI elements where the chain is not needed but we still author it for different screen resolutions.

I was about see if the library currently handles the full mipchain loading, alas, it seems that both ver1.0 and the latest version have the same bug as only the top level map is extracted. The meta info certainly provides the available mipmap info, but the data extracted is only the top level.

For your reference, I am currently tracing the code path to BasisUniversal.cs and the transcode function in BasisUniversalTranscoderInstance.cs the transcode function seems to accept to parameters which is the image index and also the mip level

It doesn't seem like the above is called at all. What is being called from the job function is the following function ktx_basisu_transcodeImage which also accepts an image index and also probably a mip level. Perhaps, the native functions are fully already full fledged out? And all that is needed is that the C# logic be correctly called?

Give me a couple of days to trace through the C# logic for the mipchain to see where the bug is. See if I can fix it and will reply over here on the status.

I am efficient in both languages so I am able to help, I code primarily on Visual Studio and do have 2017 and above installed. I don't have any cross platform toolchains installed so will need help getting those up and running including VMs if needed.

[kyapp69]

I have just completed my research and I think there could be some semantic confusion and probably a bug due to the semantics.

Upon loading a texture with mip chain:

ktx_basisu_getNumImages => reports the number of mip levels ktx_basisu_getNumLevels => and therefore consequently each image will only have 1 level

my expectation would be for _getNumImages to report 1 image and _getNumLevels to report the number of mip levels

That way if a basis file can support multiple images.. then a simple nested loop can extracted all the images with it's associated mipmapped image. ie.

foreach imageIdx in _getNumImages foreach imagelvl in _getNumLevels (imageIdx) get texturedata for each level

For now I can hack it a bit to get what I want, however, since I'm not under any immediate time constraint, it's probably better to get it correctly working for multi image support.

What are your thoughts on how to proceed?

[atteneder]

@kyapp69 Thanks for your great research!

I observed the same with my sample image (trout_mip.basis from the KtxUnityDemo project).

However reading the basisU code it seems images and levels was indeed not confused (from basisu_transcoder.h):

// Returns the total number of images in the basis file (always 1 or more).
// Note that the number of mipmap levels for each image may differ, and that images may have different resolutions.
uint32_t get_total_images(const void* pData, uint32_t data_size) const;

Next I tried to see if my sample file is indeed a basis file of one image with 11 levels or 11 images with 1 level each:

basisu -validate -file trout_mip.basis

...

Image info:
Image 0: MipLevels: 1 OrigDim: 1024x1024, BlockDim: 256x256, FirstSlice: 0, HasAlpha: 0
Image 1: MipLevels: 1 OrigDim: 512x512, BlockDim: 128x128, FirstSlice: 1, HasAlpha: 0
Image 2: MipLevels: 1 OrigDim: 256x256, BlockDim: 64x64, FirstSlice: 2, HasAlpha: 0
Image 3: MipLevels: 1 OrigDim: 128x128, BlockDim: 32x32, FirstSlice: 3, HasAlpha: 0
Image 4: MipLevels: 1 OrigDim: 64x64, BlockDim: 16x16, FirstSlice: 4, HasAlpha: 0
Image 5: MipLevels: 1 OrigDim: 32x32, BlockDim: 8x8, FirstSlice: 5, HasAlpha: 0
Image 6: MipLevels: 1 OrigDim: 16x16, BlockDim: 4x4, FirstSlice: 6, HasAlpha: 0
Image 7: MipLevels: 1 OrigDim: 8x8, BlockDim: 2x2, FirstSlice: 7, HasAlpha: 0
Image 8: MipLevels: 1 OrigDim: 4x4, BlockDim: 1x1, FirstSlice: 8, HasAlpha: 0
Image 9: MipLevels: 1 OrigDim: 2x2, BlockDim: 1x1, FirstSlice: 9, HasAlpha: 0
Image 10: MipLevels: 1 OrigDim: 1x1, BlockDim: 1x1, FirstSlice: 10, HasAlpha: 0

...

Looks like we both created basis files the wrong way :)

I'll try to create it properly and see if .basis mipmap indeed work as expected. Should be a separate issue actually.

Again, thanks for reporting!

[atteneder]

Fyi: I tried to create a mimap basis file by providing all level images (which turns out to create images instead of levels). Turns out that basis universal does support providing custom mipmaps, but only via API and not via basisu CLI (yet).

I'll fall back to using basisu's -mipmap feature for now.

[atteneder]

@kyapp69 Good news:

Once I created the mipmapped basis file correctly, it worked!

basisu -output_file trout_mip.basis -mipmap trout.png

hth

[kyapp69]

@kyapp69 Good news:

Once I created the mipmapped basis file correctly, it worked!

basisu -output_file trout_mip.basis -mipmap trout.png

My experiments showed me the same results too... So it's all good currently. At least we know that the native transcoder is reporting it correctly.

Given that. If there are any issues you would like me to help fix do say the word and the open issues and I'll be glad to help or at least give it a go.

However, version 2 of the library is not working with unity2021.2.2f1 The demo won't run. Could you see if you could do a build and run? I'm not actually concern about the demo actually running correctly as I think there are some bugs with locating the test files, however it currently crashes with some asm.js error.. I wouldn't test it against any other later versions of unity as versions of unity right up to 2021.2.6 won't build webgl for some reason.

[kyapp69]

I have modified the relevant files to support extracting a single mip level as well as creating a mipchain from a specific level. Perhaps you can integrate it into the main branch and close this issue.

The main entry function is now:

Texture2D texture = await TranscodeImage2D(transcoder, na, linear,ImageIndex,MipLevel,MipChain); //notice the addition of the last 3 parameters MipLevel is the starting level or the mip level of the image that's wanted and MipChain is whether you want to create the fullchain starting from miplevel

    async Task<Texture2D> TranscodeImage2D(BasisUniversalTranscoderInstance transcoder, NativeSlice<byte> data, bool linear, uint index = 0,uint mip=0,bool chain = true)
    {

        Texture2D texture = null;

        // Can turn to parameter in future
        uint imageIndex = index;
        uint mipLevel = mip;
        bool CreateMipChain = chain;

        var meta = transcoder.LoadMetaData();
        if (imageIndex>=meta.images.Length)
        {
            imageIndex = 0;
        }
        if (meta.images[ImageIndex].levels.Length == 1)
        {
            CreateMipChain = false;
            mipLevel = 0;
        }

        var formats = TranscodeFormatHelper.GetFormatsForImage(meta, meta.images[imageIndex].levels[0], linear);

        if (formats.HasValue)
        {
#if KTX_VERBOSE
            Debug.LogFormat("Transcode to GraphicsFormat {0} ({1})", formats.Value.format, formats.Value.transcodeFormat);
#endif
            Profiler.BeginSample("BasisUniversalJob");
            var job = new BasisUniversalJob();

            job.imageIndex = imageIndex;

            job.result = new NativeArray<bool>(1, KtxNativeInstance.defaultAllocator);
            var jobHandle = BasisUniversal.LoadBytesJob(
                ref job,
                transcoder,
                data,
                formats.Value.transcodeFormat,
                CreateMipChain,
                mipLevel
                );

            Profiler.EndSample();

            while (!jobHandle.IsCompleted)
            {
                await Task.Yield();
            }
            jobHandle.Complete();

            if (job.result[0])
            {
                Profiler.BeginSample("LoadBytesRoutineGPUupload");
                uint width;
                uint height;
                meta.GetSize(out width, out height, imageIndex, mipLevel);
                var flags = TextureCreationFlags.None;
                UnityEngine.Debug.LogFormat("<color=blue>{0}</color>", "TextureData total length:" + job.textureData.Length);
                if (CreateMipChain)
                {
                    UnityEngine.Debug.LogFormat("<color=blue>{0}</color>", "Setting mip map chains");
                    flags |= TextureCreationFlags.MipChain;

                }

                /*                
                for (int i = 0; i < meta.images.Length; i++)
                {
                    UnityEngine.Debug.LogFormat("<color=blue>{0}</color>", "Level:" + i + ":" + job.offsets[i] + ":" + job.sizes[i]);
                }

                 */
                texture = new Texture2D((int)width, (int)height, formats.Value.format, flags);
                texture.LoadRawTextureData(job.textureData);
                texture.Apply(false, true);
                Profiler.EndSample();
                UnityEngine.Debug.LogFormat("<color=green>{0}</color>", "Transcode succeeded:" + formats.Value.format);
            }
            else
            {
                UnityEngine.Debug.LogFormat("<color=red>{0}</color>", "Transcode failed");
            }
            job.sizes.Dispose();
            job.offsets.Dispose();
            job.textureData.Dispose();
            job.result.Dispose();
        }

        return texture;
    }

Changes.zip

You can close this and the relevant issue which this change affects.

Cheers

[atteneder]

@kyapp69 Fantastic! Would you mind creating a Pull Request with those changes? Did you have to adopt the native library as well? If so, another PR on the KTX-Software-Unity project would be much appreciated. I'll look at it in-depth once the PRs are in.

Thanks!

[kyapp69]

@atteneder Hi, I've just added support for separate alpha texture creation if the transcode target does not support alpha texture. ETC1 for android webgl builds. I will try to do a pull request once I cleanup the code somewhat. I'll find out how to do one of those,

[kyapp69]

@kyapp69 Fantastic! Would you mind creating a Pull Request with those changes? Did you have to adopt the native library as well? If so, another PR on the KTX-Software-Unity project would be much appreciated. I'll look at it in-depth once the PRs are in.

Thanks!

No changes to the native library at all.. All the current changes are via C# currently.

[kyapp69]

All done. Have fun and Merry Xmas.

[atteneder]

Thanks <3 I'll have a look after the holidays. Merry Xmas as well!