Offer more control for the binary/embedded conversions

[javagl]

There is some sort of ambiguity in what exactly constitutes a "binary", "embedded" or "default" glTF model.

Technically, it is possible to create a binary glTF that 1. contains references to external files in its JSON part, and 2. contains resources embedded as data URIs in its JSON part. This may or may not be desired.

Right now, the GltfAsset structures are a "low-level"-representation of the assets, and as such, allow "mixed type assets". When writing a GltfAsset, it is written as-it-is, without doing any conversions. For example, when a GltfAsset that contains embedded resources (as data URIs) is written as a binary glTF, then the JSON part of the binary glTF will still contain the data URIs.

In contrast to that, the GltfModel classes serve as an abstraction layer that does no longer differentiate between these types. When a GltfModel is written, then all its resources are written in the desired format (default, embedded or binary).

Internally, the GltfModel is converted into a GltfAsset before it is written, and this GltfAsset will contain resources only in the desired target type. But there should be an API that allows a fine-grained control over the conversion. For example, such an API could/should offer options to convert resources into binary/embedded based on their type (i.e.: "Write all images as binary, but all buffers as embedded"), or to convert them individually (i.e.: "Write the first image as embedded, and the second as binary").

[Trass3r]

should offer options to convert resources into binary/embedded based on their type

Exactly, I want to reference image files but embed buffer data, at least for small buffers.

[javagl]

@Trass3r What is a 'small' buffer ? 😉

There are some ideas floating around (although only very abstractly, in my head).

The GltfModelWriter.write... family of functions essentially create a GltfAsset internally, using the corresponding ...AssetCreator classes (e.g. BinaryAssetCreatorV2), and just write that out as it is. And it wouldn't be soooo hard to create a first shot of some CustomAssetCreator. There are some questions about the degree (and granularity) of the configurability. For example, regarding the "small" buffers: It's certainly reasonable to make certain decisions based on the size of the resource, and once could just go ahead and introduce some pragmatic Predicate, like c.setBufferEmbeddingPredicate((b) -> b.size() < 1000); But there are many more degrees of freedom, and it may require a bit more thought.

Until now, there didn't seem so much demand for that, but maybe I can allocate some time to draft something here...

(The nice thing could be that if this is done right, then this CustomAssetCreator could replace all the existing ...AssetCreator classes, which could then just be instances with predefined configurations....)

[Trass3r]

Yeah not sure either. I'm writing a gltf exporter and need to inspect the output a lot. So big base64 blobs are just in my way. And embedding images doesn't make sense at all. (And user-defined names for everything would be very helpful, they seem to exist in the low-level classes only). Keeping at least small buffers embedded produces less files and might make it easier to see/modify the content. Though even https://marketplace.visualstudio.com/items?itemName=cesium.gltf-vscode doesn't support inspecting/modifying them directly, so not sure if it makes any sense to try at all. But the standard gltfWriter.write doesn't work anyway, some NullPointerException.

The GltfModelWriter.write... family of functions essentially create a GltfAsset internally

Yeah I hacked it a bit, inlining

var gltfWriter = new GltfModelWriter();
gltfWriter.writeEmbedded(gltfModel, outputFile);

and then

var gltfWriter = new GltfModelWriterV2();
gltfWriter.writeEmbedded(gltfModel, outputStream);

resulting in

GltfAssetV2 gltfAsset = GltfAssetsV2.createEmbedded(gltfModel);
GlTF gltf = gltfAsset.getGltf();
int i = 0;
for (var image : gltf.getImages())
  image.setUri(gltfModel.getImageModel(i++).getUri()); // reset data: uri to file reference

var gltfWriter = new GltfWriter();
gltfWriter.write(gltf, outputStream);

[javagl]

Keeping at least small buffers embedded produces less files and might make it easier to see/modify the content. Though even https://marketplace.visualstudio.com/items?itemName=cesium.gltf-vscode doesn't support inspecting/modifying them directly, so not sure if it makes any sense to try at all.

The VS Code plugin does allow inspecting the data - right-clicking an accessor and selecting "Inspect data":

I use this regularly. (One inconvenience is that VS Code can not open GLB files direcly - they have to be converted to .gltf, by right-clicking them and "Import from GLB"...)

The idea of "inspection and browsing" was also the reason for the JglTF glTF Browser - see the screenshots at the bottom of https://github.com/javagl/JglTF/tree/master/jgltf-browser (But ... the browser has not been updated/maintained for quite a while, and it does not have a "proper" glTF 2.0 (PBR) renderer. Too much work, too little time (and incentive, beyond "it would be cool"...)).

Editing (i.e. modifying) the values is a different question. There's https://gestaltor.io/ (but I haven't extensively tried that, to be honest), and you can always load glTF in Blender and edit it there (but ... Blender is a beast with a steep learning curve).

A ... somewhat adventurous way of "editing (small) glTFs" that I've been using occasionally is https://github.com/javagl/gltfTransformifier . This project is only a very basic and experimental proof of concept, but ... maybe I'll try to find some time to extend it. The approach there is: You pass in a glTF file to the GltfTransformifier, and it will generate the source code of a TypeScript file that generates the given glTF asset, with glTF-Transform. So you'll have some auto-generated code like

  // Accessor 8 of 10
  const accessor8 = document.createAccessor('');
  accessor8.setType('SCALAR');
  accessor8.setBuffer(buffer0);
  accessor8.setArray(new Float32Array([
    0, 
    1.25, 
    2.5, 
    3.708329916000366
  ]));

where you can edit the values, and then run that file, and it will generate the glTF with the modified values.

(Yeah, that's not "convenient", but it's unbelievably powerful...)

(And yeah, I should create the same thing for the https://github.com/javagl/JglTF/tree/master/jgltf-model-builder ... when I have time...)

But the standard gltfWriter.write doesn't work anyway, some NullPointerException.

I know that there are some cases/configurations where reading/writing assets and converting between default/embedded/binary do not work 100% smoothly. I'll also have to allocate more time to create some test coverage with "roundtrips" between all configurations, including the "mixed" ones that are currently rare (e.g. embedded buffers but external images and such). But if you have a specific file or workflow that causes problems here, then it could make sense to track that in a dedicated issue.

[Trass3r]

And user-defined names for everything would be very helpful, they seem to exist in the low-level classes only

Turns out they are there. I can set them manually just fine. Also improves the view in Babylon's scene inspector. It's a pity you can't set a name on mesh primitives, can be emulated with extras though. But the names for the accessors/buffer views/buffers that are getting created under the hood when you do something like

            var meshPrimitiveBuilder = MeshPrimitiveBuilder.create();
            meshPrimitiveBuilder.setByteIndices(...);
            meshPrimitiveBuilder.addPositions3D(...);
            meshPrimitiveBuilder.addTexCoords02D(...);
            var meshPrimitiveModel = meshPrimitiveBuilder.build();

are hard to set.

I worked around it a little bit with

            for (var accessorModel : meshPrimitiveModel.getAttributes().values())
                ((DefaultAccessorModel) accessorModel).setName(myDebugName);
                // interestingly accessorModel.getBufferViewModel() is null

and on the DefaultGltfModel:

        for (var accessorModel : gltfModel.getAccessorModels()) {
            var bufferViewModel = (DefaultBufferViewModel)accessorModel.getBufferViewModel();
            bufferViewModel.setName(accessorModel.getName());
            // doesn't make too much sense, contains lots of different data
            var bufferModel = (DefaultBufferModel)bufferViewModel.getBufferModel();
            bufferModel.setName(accessorModel.getName());
        }

[javagl]

Regarding the names: There's a TODO at

https://github.com/javagl/JglTF/blob/8dfc2a99313ee615150989ccad400a8b28af6ef1/jgltf-model-builder/src/main/java/de/javagl/jgltf/model/creation/DefaultBufferBuilderStrategy.java#L53

Yeah, TODOs in code are usually bad, and in doubt, they should be converted into issues. Until now, this one didn't seem a toooo pressing issue, but if necessary, this could become an issue, to investigate some options about how to tackle that.

[Trass3r]

The VS Code plugin does allow inspecting the data - right-clicking an accessor and selecting "Inspect data":

Yeah seems like it does work with accessors but I tried on a buffer or buffer view, don't remember, and got a message that isn't supported.

if you have a specific file or workflow that causes problems here, then it could make sense to track that in a dedicated issue.

Might be that Im using the API incorrectly but I didn't investigate further since writeEmbedded worked.

[javagl]

I tried on a buffer or buffer view, don't remember, and got a message that isn't supported.

Well... the buffer view does not have any type information, so you could only look at the plain bytes (and for something like a buffer with floats, that's not really helpful).

[javagl]

On the one hand, this has largely been addressed via https://github.com/javagl/JglTF/pull/108

But the actual configuration endpoints have not yet been exposed. So I'll leave this open for now...