An option to keep separate buffer views when doing `meshopt`

[mixtur]

Is your feature request related to a problem? Please describe. Here is the way we intend to use glTF transform. We do a number of operations using @gltf-transform/cli, then translate resulting glTF into our format. By doing it like that we offload heavy lifting in optimizations to glTF-transform, while also having a format that suits our needs better than glTF.

Now the problem. When using meshopt command: A. It seems glTF-transform merges bufferViews and then compresses them with meshoptimizer. The problem is that in our format we store data for each geometry in its own chunk. But it is impossible when a bufferView is shared across multiple primitives. B. meshopt command doesn't respect --vertex-layout option. Regardless of the layout in the option, vertex layout is interleaved. This is not exactly a "problem" for us, but it would be nice if it was fixed,

We only checked meshopt, but maybe it happens with other operations as well.

Describe the solution you'd like For problem A - a way to force glTF-transform to keep bufferViews. Or even better - to make bufferViews to correspond to exacly one primitive (except when it would produce duplicate bufferViews) For problem B - respect --vertex-layout option. I believe meshoptimizer can hande deinterleaved attributes.

Describe alternatives you've considered We could decode meshopt, fix both problems on our side, and then compress everything back, but it kinda defeats the purpose of using glTF-transform.

Another alternative would be to use gltfpack but it also seems to merge bufferViews (slightly dfferently than glTF-transform btw, which is curious)

[mixtur]

It seems using partition command and then meshopt command produces separate bufferViews (and separate buffers) per mesh. But our models are VERY large (think ~10000 meshes or more) and for those partition works very slowly. I believe it does something superlinear in number of meshes. So this is not practical for us.

[donmccurdy]

Thanks @mixtur! Both gltfpack and glTF-Transform impose preset buffer view layouts, rather than attempting to maintain existing buffer view layouts as-is. glTF Transform keeps buffer assignments, and offers two document-level presets (interleaved and separate) for buffer view assignments, as you've observed, with some special cases for Draco and Meshopt compression. It would be much more complex to do any significant optimization while keeping arbitrary buffer view layouts intact.

About meshopt not organizing accessors by mesh, as the uncompressed output would... The relevant code is here:

https://github.com/donmccurdy/glTF-Transform/blob/a9811deb76baf27fe3ecdda696a5277d73dbb179/packages/extensions/src/ext-meshopt-compression/meshopt-compression.ts#L311-L319

For problem B - respect --vertex-layout option.

I agree it would be better for meshopt to match the default uncompressed VertexLayout settings as closely as possible, grouping accessors into buffer views by mesh primitive. If there's need for buffer views not to be grouped my mesh primitive, a new VertexLayout option could be added for that.

However, note that 1:1 mesh primitive to buffer views mappings are not guaranteed. Different vertex attributes need different filter settings, which require different buffer views, and vertex attributes may potentially be shared across mesh primitives, in which case the choice of which bufferview the attribute is stored in becomes arbitrary. If you're sure the document doesn't reuse attributes across primitives, then I think you can assume it's either a 1:1 or 1:many relationship.

Does this sound like it would work for you?

But our models are VERY large (think ~10000 meshes or more) and for those partition works very slowly. I believe it does something superlinear in number of meshes.

This can probably be fixed - do you mind opening a new issue for it, with details and a sample model if possible?

[mixtur]

It would be much more complex to do any significant optimization while keeping arbitrary buffer view layouts intact.

So problem A is out of the question?

Does this sound like it would work for you?

Yes. I guess "--vertex-layout interleaved" should produce "interleaved" result and "--vertex-layout separate" - "separate". If there will be some additional bufferViews it still counts as "separate" IMO.

This can probably be fixed - do you mind opening a new issue for it, with details and a sample model if possible?

I was sure I did it like 2 days ago. But indeed the issue is not there. Huh. I'll make sure I actually open it this time :)

[donmccurdy]

So problem A is out of the question?

The in-memory representation of the 3D model in glTF Transform does not have a concept of bufferviews — just the assignment of Accessor#setBuffer, see concepts. This is by design — glTF is extremely flexible about bufferview layouts, and I believe trying to build optimization features on top of unknown input bufferview layouts would be impractical.

So instead, bufferviews are assembled during export, based on the VertexLayout option. I'm open to the idea of adding more options, but it sounds like just having meshopt respect the existing options is what you need? For Draco there isn't much choice, but I agree this should be changed for Meshopt. 👍🏻

[mixtur]

I guess I was wrong to describe two problems in one issue. But those are two separate things.

Problem A:

We have a converter from gltf to our format. Our format allows progressive loading. We show more of a model as we load it.

To do that we separate whatever huge input model to smaller chunks and serve that. In the perfect world we would be able to separate the original model on primitive boundaries. When bufferViews are not compressed we can still do it. But if bufferViews are compressed, as it happens with EXT_meshopt_compression, the best we can do is to separate the input model on bufferView boundaries. Which produces chunks that are too large.

One alternative would be to apply meshopt to uncompressed meshes ourselves. But since gltf-transform is already there we thought we could use it.

If two-step process with partition -> meshopt works in reasonable time, this is enough for us. If you're willing to add an option to apply meshopt per-primitive and store result in separate bufferViews - that would also be ok.

Problem B:

Currently meshopt doesn't respect --vertex-layout.

This was just a side observation that we discovered when exploring Problem A. It would be nice if it was fixed, but it is not really important for us. I would probably be satisfied even if there was a warning that says "Sorry --vertex-layout is not implemented for meshopt command".

[donmccurdy]

With #1384 glTF Transform will split buffer views by primitive when doing Meshopt compression, as it does when writing uncompressed primitives. In testing this didn't seem to have much effect on file size, other than the cost of declaring the additional buffer views.

[mixtur]

Tried 4.0.0-alpha.18 - it works! Thank you a lot.