C/C++ code for viewing animated GLTF 2.0 (with skinning+morphing) ?

[cesss]

Hi!! First of all, thanks a lot for your great work not only in tinygltf but in all your projects!! Looking at the viewer included in tinygltf, I see that animation is in the TODO list. I'm starting a project (scene viewer/player in C/C++, not limited to just GLTF but fully supporting it) which in the short/mid-term future (some months from now) will need skinning+morphing animation.

Rather than reinventing the wheel, I believe it's a wise choice to build on top of an already existing 3D scene library that implements skinning/morphing. But I don't want one of those huge libraries that suffer from a dependency hell: I want it very minimal, very simple, with no dependencies (or just a few), written in C++ or C, and with a permissive license (MIT or a similar one) because I want my project to be distributed with a permissive license.

I suppose you have already considered these topics while you thought in adding animation support to your viewer, so, may I ask if you found any "tiny-like" 3D library with skinning and morphing that could support all the animation features in GLTF 2.0, so that it could be connected to your tinygltf library as a rendering backend?

Today I've been taking a look at the Godot engine, because it seems to build with almost no dependencies, seems to support skinning and morphing, and has a permissive license, but maybe it's overkill because it's a complete game engine, and I don't need game logic code.

Did you find any other interesting code for implementing animation in your viewer, or maybe you were planning to write the code from scratch?

Thanks!!

[syoyo]

There is work-in-progress simple/minimal skinning implementation branch

https://github.com/syoyo/tinygltf/tree/skinning/examples/skinning

You may start implementing skinning and morphs from here.

FYI, https://github.com/SaschaWillems/Vulkan-glTF-PBR will be a good reference.

And PR is always welcome!

[cesss]

Thanks a lot! I'm having problems compiling Vulkan-glTF-PBR on Mac (yes, I know MacOS is not listed as one of the supported platforms, but I'm almost there, the Vulkan SDK is successfully detected by CMake..., I'm going to open an issue in the project repository because I feel it might be very easy to add MacOS support to it).

[Ybalrid]

@cesss I'm not sure if this is now packaged with the official VulakSDK by LunarG (should be now), but on MacOS you will need https://github.com/KhronosGroup/MoltenVK

Vulkan itself isn't available on Mac OS. However, MoltenVK provide a translation layer that converts calls to the Vulkan API into calls to the Apple "Metal" API.

All the data for reading skining and morphing animations are available through tingygltf, and are exposed as the glTF documentation show them. There's a handy reference poster here: https://raw.githubusercontent.com/KhronosGroup/glTF/master/specification/2.0/figures/gltfOverview-2.0.0b.png

If you need a few pointer, an animated mesh that has "skins" will point to a sub-tree of the scene. These nodes are the joints of the object's skeleton. Animations that uses them will move the transform of these nodes. In the case that an object has skinning animation, you can look for "JOINT_0" values in the data that describe the primitives of the mesh, they indicate for each vertex what joint (bone) will influence it's actual position in model space.

Morph animation generally points to "sparse" accessor to the same "vertex buffer" the object has. These describe only the small changes inside the whole vertex array. They indicate the index of values to update, and their new values, instead of having a copy of the whole array. In a tinygltf "accessor", there's a handy "isSparse()" method that returns true if you need to check the "sparse" property for these changes.

Animations Samples are simply successions of "key frames". The accessor pointed as "input" contains their timepoint, the accessor as "output" contains their values for each corresponding timepoint. You need to interpolate between the two keyframes using the method described in the sampler.

I have a (skining only) implementation of this using Ogre 2.1 (a 3D rendering engine), but it may help you loading the keyframes of animations, and apply them to a skeleton : https://github.com/Ybalrid/Ogre_glTF/blob/master/src/Ogre_glTF_skeletonImporter.cpp

[cesss]

Thanks a lot, @syoyo and @Ybalrid !! It would be great to see the tinygltf viewer animation branch merged soon!!

Btw, are you aware of any (open source) skeletal animation editor (apart from Blender) in which for example you could load mocap files, set/tune the "resting" skeleton pose, blend animations, retarget them, etc? I remember that many years ago, people used Milkshape3D for tasks like these, but I'm searching GitHub and found nothing (well, there's https://github.com/DaveDubUK/bvhacker, but it's Windows-only...).

I'm asking this because if there was such an editor, it could be a good starting place for adding GLTF2 support to it and base my work on top of its data structures...

[syoyo]

@cesss @Ybalrid is now working on implementing skinning/blendshapes, and animation debugger in gltf-insight https://github.com/lighttransport/gltf-insight . Functional version will be released around the end of Jun.

Then, you can contribute such a feature to gltf-insight! > load mocap files, set/tune the "resting" skeleton pose, blend animations, retarget them, etc?

[cesss]

That project is awesome!! Yes, it would be a good place to start!!

[Ybalrid]

@cesss For now it's really early in development. Here's a sneak peak of the displaying of the bones inside a skinned glTF mesh :wink:

[cesss]

Very, very promising!!!!

[syoyo]

gltf-insight now has enough feature to view skin + morph. So close the issue