Extension support

[javagl]

The topic is as broad and generic as the title suggests. Right now, there is not really support for any extensions on the level of the jgltf-model classes. There is support for some extensions on the level of the ...impl classes, which generally contain the plain old Java structures that are auto-generated from the respective JSON schemas (and the infrastructure for this is already in place). But these extensions can now, at best, be passed through via the untyped extensions in the model classes.

Proper support for extensions on the level of 'model' classes raises a whole bunch of engineering challenges.

The lowest level is in establishing the connection between the 'impl' and 'model' classes. In any cases, this means, very roughly speaking, to replace indices with objects. Instead of

gltf.extensions["KHR_lights_punctual"].lights = /* let there be lights */;
node.extensions["KHR_lights_punctual"].light = 4; // The index

the model classes should allow something like

Light light = new ...;
gltfModel.getExtension(KHR_lights_punctual.class).addLight(light);
nodeModel.getExtension(KHR_lights_punctual.class).setLight(light); // The object

Parts of that are pretty straightforward. But that pseudocode involving KHR_lights_punctual.class already hides the difficult parts: There has to be some sort of "plugin concept" for extensions, so that the model classes can be used when the respective model classes JAR is found on the classpath.

The next level is that of how the extensions affect other parts of the code.

This may include seemingly trivial things. For example, the auto-generated impl classes currently enforce the contraints that are defined in the JSON schema. For example, when trying to do accessor.setByteoffset(-123), then this will throw an IllegalArgumentException. Now... there might be some OBSCURE_accessor_negative_byte_offset extension that allows negative byte offsets. How to handle something like that is all but clear.

Far more tricky is the question of how the presence of extensions affects reading, writing, and processing the model. I assume that adding support for extensions will involve many "breaking changes", insofar that there will very likely be some doSomethingForExtensions(...) methods that have to be added in interfaces or classes, to offer some form of "hooks" or "callbacks" that allow certain forms of pre- and post-processing.

I'll probably start some experiments with the most simple extensions (like KHR_lights_punctual or some of the PBR material extensions that just add some textures or floating point values). But extending the support for extensions will likely be an iterative process that spans over several versions/releases.

All this is still independent of some of the really difficult questions for specific extensions. Good luck with trying to find any support for Draco or MeshOpt in the Java world.

(There is some hope for KTX, at least - via https://github.com/KhronosGroup/KTX-Software/pull/886 . But ... my spare time is limited, and at the end of the month, I have to pay my rent...)

[bchapuis]

Came accross this note when looking for draco compression in JglTF. I wonder if the wasm distribution of draco could be used in the JVM with GraalWasm.

https://www.graalvm.org/latest/reference-manual/wasm/

[javagl]

From scrolling over that intro page from GraalVM, it looks pretty straightforward. I know that WASM in TypeScript/JavaScript can be a pain in the back, due to the usual quirks (What is a 'module'? What does export mean? What is the default export? Who messed this up so badly?). But ... this is Java, so there's hope. I'll try to allocate some time to try it out. (But have to juggle tasks and priorities, with JglTF as a whole, things like the KTX JNI bindings, and faaar to many other "spare time projects" that I try to squeeze into my "spare time").

[bchapuis]

No emergency on my side ;) I will let you know if I find time to experiment as well (libktx also compiles to wasm).

[javagl]

The usual concern with WASM is about performance. I once did a quick test for KTX, comparing the native toktx with a TypeScript/WASM version, at https://github.com/donmccurdy/glTF-Transform/issues/675#issuecomment-2007340429 (ONLY a quick test - to be taken with the usual grain of salt), and this suggests that WASM may be "significantly" slower. How important this is in practice, and how to weigh in the usual tradeoffs, is hard to say.

In the Java world, the trade-off would be between WASM and JNI, and for KTX, there are currently efforts for consolidating the latter. For Draco, it's different. I'm not deeply involved here, and don't know how feasible JNI bindings would be. (I actually have some C:\Develop\JDraco directory here, where I naively thought "Hey, I'll just port that code to Java", but... not another "spare-time project"...). The WASM approach could be a relatively easy win here.

[Jeongyong-park]

There is a KTX jni binding library, but I don't know if this will help.

https://github.com/KhronosGroup/KTX-Software/tree/main/interface/java_binding

[javagl]

@Jeongyong-park The current state of the KTX Java Bindings is ... ... ... ... has a lot of room for improvement. That's the reason why I created https://github.com/KhronosGroup/KTX-Software/pull/886 . But this is not merged yet.