Support glTF extension: EXT_meshopt_compression

[mvaligursky]

extension specification: https://github.com/KhronosGroup/glTF/tree/master/extensions/2.0/Vendor/EXT_meshopt_compression

library https://github.com/zeux/meshoptimizer

this should work alongside already supported draco compression to allow engine to load both compressed mesh formats.

[mvaligursky]

I've done some investigation, here's some findings:

1. We have implementation of EXT_meshopt_compression in the viewer: https://github.com/playcanvas/playcanvas-viewer/blob/72e23121ea4d83f60a32d83d13abf3b474be649b/src/viewer.ts#L612-L637

2. we have partial implementation of KHR_texture_transform, see here: #2563

3. we do not have implementation of KHR_mesh_quantization, see here: #2636

4. Size and performance. I compared uncompressed, draco compressed and meshopt compressed glbs, and the results are very promising. Size: draco is smallest, with meshopt being close second, uncompressed being distant third. Loading speed: draco is slowest (5x time slower than uncompressed), meshopt is fastest (usually about 20% faster than uncompressed).

Table shows loading times (in ms, first line) and size (second line) for test glbs.

mesh	draco	meshOpt	uncompressed
bed	655	85	105
	2.1MB	3.1MB	24MB
bitmoji	78 (no morph)	113 (broken morph)	150
	3MB	0.2MB	4MB
ChocoBunny	37	6	16
	48k	91k	465k
Lion	70	23	26
	1.5MB	1.6MB	2.9M
Iris	36	8	19
	105k	54k	450k
Heart	7	6	6
	4.7k	5.6k	15.4k

What needs to be done to fully support this:

1. move EXT_meshopt_compression support into the engine directly. The integration code from viewer is small. The decoder itself (https://github.com/playcanvas/playcanvas-viewer/blob/master/lib/meshopt_decoder.js) is 26kb uncompressed, this can be optionally added to project when user requires this extension.

2. When gltfpack (tool to encode glb using meshopt, https://github.com/zeux/meshoptimizer/tree/master/gltf) compresses data, it applies quantization to many (all) elements (for example position is stored in INT16 format and similar). To scale these values, KHR_texture_transform is used. For position itself, the world matrix of the glb node is adjusted, and so this is handled automatically. We have problem with quantized morph targets - as those are not currently scaled, and so end up very incorrect.

[mvaligursky]

For testing purpose - animated morph cube, but uncompressed and also meshopt compressed. Compressed version does not work in the viewer as explained above. AnimatedMorphCube.zip

[Maksims]

meshOpt - sounds like a very promising fast parsing solution here. Even faster than non-compressed. Would be interesting to see compression speeds as well, how long it takes to compress geometry, and what are the tooling available out there. Is there command-line tools available, and how fast they are comparing to each other.

[mvaligursky]

I think part of the reason its faster than uncompressed is the fact it uses SIMD in browser supports if (and Chrome on Mac I think does already if you apply for access), and also the fact that it keeps some attributes quantized - using int16 instead of float inside VBs and it uses transform to unpack it. But maybe it's something else as well.

Regarding tools - I didn't test performance of encoding, as that's the cost we'd accept for better runtime performance. For reference, I used:

• compress to draco: https://github.com/CesiumGS/gltf-pipeline
• compress to meshopt: https://github.com/zeux/meshoptimizer/tree/master/gltf

[marklundin]

These numbers sound awesome. Am I correct in saying that if the viewer code is ported over the engine, it would support meshopt geometry, but only those without morph targets?

[mvaligursky]

Morph targets is the one thing I confirmed that didn't work out of the box, but I suspect some other streams if quantization is used might not work either .. but I didn't hit this issue with few meshes I tested and the default compression settings.

The viewer just registers the decoding part as a preprocess option on the glb-parser for this to work. You can test this out in your project as well .. just copy those 30 or so lines and it should work. And decoder wasm module.

[br-matt]

This would be greatly appreciated, we are thinking of switching over to playcanvas and using meshopt + ktx compressed assets that come via https://github.com/zeux/meshoptimizer/tree/master/gltf

[mvaligursky]

Perhaps try some compressed models here in our viewer: https://playcanvas.com/viewer

and see if you hit any issues or not.

[br-matt]

Hey, I gave it a try and no luck. I got a different error than when trying locally with engine.

I am able to load the uncompressed version of the asset both locally and that viewer.

If you run any glb/gltf through gltfpack, use the -cc -tc options to replicate

[mvaligursky]

I don't believe we have support for compressed texture extension KHR_texture_basisu, so try -cc only, without -tc

[br-matt]

Thanks, it does indeed load with just -cc but then KHR_texture_transform looks like it gets applied incorrectly making it look something like a mc escher painting :(

Any plans to support KHR_texture_basisu in the future ? I saw playcanvas supported ktx2 textures so thought it had to have already somewhere.

[mvaligursky]

We have not had any progress on KHR_texture_transform yet unfortunately. https://github.com/playcanvas/engine/issues/2563

@slimbuck - any idea on complexity of adding support for KHR_texture_basisu?

[slimbuck]

Should be relatively straightforward to add KHR_texture_basisu support.

Added https://github.com/playcanvas/engine/issues/2935

[mvaligursky]

some good comments here: https://forum.playcanvas.com/t/tricks-to-decrease-morph-target-sizes/18628/12

if -noq option is used while compressing to meshopt format, no quantization is used and our viewer can probably load it including compressed morph targets.

[mvaligursky]

a thread discussing how to move viewer's meshopt support into the engine: https://forum.playcanvas.com/t/compressed-glb-file-is-not-loading-in-the-editor/19220/9

[mvaligursky]

extra data point regarding the Draco decoder performance (can be improved by 2x by preallocating memory) https://github.com/google/draco#javascript-decoder-performance

[willeastcott]

That's cool - but we can only practically ship 1 build of Draco. Devs can rebuild their own version with the static memory tuned to a sensible size - but that's not very convenient.

[donmccurdy]

FYI: https://github.com/playcanvas/playcanvas-viewer/issues/125

^This issue comes from using a different tool to encode with EXT_meshopt_compression; unlike gltfpack this does not currently require KHR_texture_transform if that's helpful for testing. I haven't added a CLI flag for disabling KHR_mesh_quantization but could do so if needed.