Any interest in shared KTX 2.0 parser?

[donmccurdy]

Discussed this briefly with @bghgary recently, but I think the current state of (non-WASM) KTX 2.0 parsers for web is:

• three.js has a custom plain JS parser, THREE.KTX2Loader. Parses KTX2 wrapper and decodes using MSC Transcoder.
• babylon.js has a TypeScript port KTX2 Decoder. Parses KTX2 wrapper using combination of MSC Transcoder and lightweight AssemblyScript transcoders.
• (new) I've started a TypeScript KTX2 parser + serializer, ktx-parse. It's a standalone repository with an npm package, so that I can use it in other projects like glTF-Transform. My intention (open to discuss) was to keep it focused on parsing the container, not transcoding, so that it could be used with any of the various Basis transcoders or other GPU texture formats.

three.js does not bring in production dependencies from NPM, so my ability to package that code for reuse is a bit limited. If you're interested though, I'd be glad to collaborate on a repo/package for the other two, and for future users. Currently the code I've written in read.ts is very similar to your own ktx2FileReader.ts, and outputs a nearly-identical container interface. I've added support for key/value data, otherwise I think they're functionally equivalent. I intend to include serialization as well (not functional yet), which will be tree-shakeable if you don't need it:

import { read, write } from 'ktx-parse';

// Load.
const data = await fetch('./input.ktx2').then(r => r.arrayBuffer());

// Parse.
const container = read(new Uint8Array(data));

// Serialize.
write(container); // → Uint8Array

Open to ideas on how to structure this, or let me know if this doesn't make sense. :)

[sebavan]

As we are also not having dependencies, I ll definitely let @bghgary and @Popov72 think about this one, but I truly enjoyed the shared aspect and community focus of the proposal !!!

[bghgary]

Yeah, we don't bring in npm dependencies either. Would it be possible to create a standalone js file that we distribute? For example, Draco has the js file directly commited in their repo and we just copy it. Another example is glTF-Validator where we use browersify to build a standalone js.

That is what we were thinking for the KTX2 Decoder. It would live in its own repo and we would bring in a pre-built js file. Open to other ideas though.

[Popov72]

Currently the code I've written in read.ts is very similar to your own ktx2FileReader.ts

That's expected as we heavily based this implementation on your own implementation from 3js KTX2Loader.js (this is our first reference in the header comment of ktx2Decoder.ts) ;)!

[donmccurdy]

Would it be possible to create a standalone js file that we distribute?

That's easy if it's just a parser+serializer, yes. If scope includes transcoders, ZSTD or ZLIB decompression, and Web Worker management, then those could add several extra JS+WASM files that need to be copied around in the right directory paths, and things get more complicated.

[bghgary]

I don't think we should include web worker management. I think this should be handled by the consuming code.

I also don't think we should include the transcoders or ZSTD directly. Instead, we can use configuration options to point to them or add them via some kind of extensibility mechanism. It will be up to the consuming code to determine where to host the additional assets (wasm, etc.).

[donmccurdy]

I've finished up the reading/writing functionality of ktx-parse now, and this quick transcoding test works:

const fs = require('fs');
const savePixels = require('save-pixels');
const ndarray = require('ndarray');
const {read} = require('ktx-parse');

const decoderWASM = fs.readFileSync('./node_modules/universal-texture-transcoders/build/uastc_rgba32_srgb.wasm');
const ktxData = fs.readFileSync('./test_uastc.ktx2');

(async () => {
    const texture = read(ktxData);

    const {byteLength} = texture.levels[0].levelData;

    // Uncompressed texture padded to multiple-of-4 height
    const yBlocks = (texture.pixelHeight + 3) >> 2;
    const uncompressedByteLength = texture.pixelWidth * yBlocks * 4 * 4;

    const texMemoryPages = (65535 + byteLength + uncompressedByteLength) >> 16;
    const memory = new WebAssembly.Memory({ initial: texMemoryPages + 1 });

    const decoder = (
        await WebAssembly.instantiate(decoderWASM, {env: {memory}})
    )['instance'].exports;

    const compressedTextureView = new Uint8Array(memory.buffer, 65536, byteLength);
    compressedTextureView.set(texture.levels[0].levelData);

    const decodedTextureView = new Uint8Array(memory.buffer, 65536 + byteLength, uncompressedByteLength);
    const status = decoder.decodeRGBA32(texture.pixelWidth, texture.pixelHeight);

    if (status !== 0) throw new Error('Decoding failed.');

    saveTofile('output.png', decodedTextureView, texture);
})();

///////////////////////////////////////////////////////////////////////////////

async function saveTofile(path, view, texture) {
    const pixels = ndarray(view, [texture.pixelWidth, texture.pixelHeight, 4])
        .transpose(1, 0)
    const image = await new Promise((resolve, reject) => {
        const chunks = [];
        savePixels(pixels, 'png')
            .on('data', (d) => chunks.push(d))
            .on('end', () => resolve(Buffer.concat(chunks)))
            .on('error', (e) => reject(e));
    });
    fs.writeFileSync(path, image);
}

I didn't include ZSTD decompression in the test, but it should be minor to add:

import { ZSTDecoder } from 'zstddec';
const decoder = new ZSTDDecoder();
await decoder.init();

// ...

if (texture.supercompressionType === 2) {
    data = decoder.decode( data, texture.levels[i].uncompressedByteLength );
}

The build output is https://unpkg.com/ktx-parse@0.0.4/dist/ktx-parse.modern.js, and would benefit from tree-shaking if you don't need to write KTX2 files. Does this fit what you had in mind? I

[bghgary]

Does this fit what you had in mind?

Thanks, it seems good from a quick look! @Popov72 What do you think?

[Popov72]

That seems fine to me!

I'm wondering however if we should not also provide (afterwards) another package like ktx-transcode (or ktx-decode) that would use ktx-parse to decode files (what the 3js KTX2Loader and Babylon ktx2Decoder do), because people needing ktx2 decoding will end up writing basically the same thing. That's something we thought about on our side when creating ktx2Decoder and that's why we did a separate package from Babylon.js itself with no dependency, in case we would create a standalone repo later on.

One small thing: why not ktx2-parse instead of ktx-parse? Do you support both ktx and ktx2 file formats?

[donmccurdy]

One small thing: why not ktx2-parse instead of ktx-parse? Do you support both ktx and ktx2 file formats?

I went back and forth on that. Support for reading KTX 1 and writing KTX 2 might be nice (and relatively easy?), but isn't a high priority for me at the moment. I also didn't want to rule out supporting future versions of KTX, and to some extent I'm hoping that "KTX" will eventually be synonymous with "KTX 2", similar to glTF, even though there's a longer history there.

I'm wondering however if we should not also provide (afterwards) another package like ktx-transcode (or ktx-decode)

It feels more difficult to make that fully reusable, for a couple reasons:

• KTX 2 does allow compressed texture formats other than Basis Universal – the parser works regardless, but a transcoder or decoder would need to decide how much is in scope.
• Which transcoder(s) should be supported?
• Support Node.js, or just Web? Support non-WASM browsers?
• Use Web Workers, or leave that to the end user?

Not to say it wouldn't be useful, but there are more difficult decisions on the transcoding side.

[Popov72]

I agree there would be some decisions to be made, that's why we decided to create ktx2Decoder inside Babylon.js and not as a standalone package for the community to use, to avoid delay in implementation as we needed it quickly. However, even if restricting the scope, I think it would still be useful for a number of people/projects. Also, we can always add new features as time goes as long as it does not break backward compatibility. So it could start relatively small in scope:

• Web only for WASM browsers - I don't know what has to be done to support node.js, but if not too much a hassle the support could be added at start
• Basis Universal
• Use Universal Transcoders when possible and MSC Transcoder else (as the universal transcoders are much faster)
• No web workers in this lib. The lib is only a step above ktx-parse and add the transcoding processing, it's up to the higher level to setup a pool of workers if necessary.

That's basically what we did in ktx2Decoder.

[donmccurdy]

We've included ktx-parse in three.js as of r125, for reading .ktx2 textures. In theory writing is also fully supported but I haven't done as much testing on that yet. At least for now we are putting a custom wrapper around it, since we want to use abstractions like THREE.FileLoader to fetch WASM binaries and such.

[donmccurdy]

Closing this, since it is more of an open-ended idea than an actual issue. The repo is currently at https://github.com/donmccurdy/KTX-Parse, if you would like to use it and are having any issues let me know!