Sharing skeletons between two skinned mesh

[bghgary]

This is a spin off from #1195 to discuss how to properly share skeletons between two skinned mesh.

For a nice summary of this issue, see @donmccurdy's comment here.

We realize this doesn't allow glTF 2.0 to support several features that are important for skinned animation, but the current spec cannot support them now without breaking changes. From @Nehon's earlier comments:

1. Re use of a skeleton for multiple meshes
2. Partially skinned meshes
3. Forces to bake a skinned mesh’s animation into the root bone’s animations

We can start discussing how to make skinned animation better in this issue.

Here are some considerations:

• Can this be done in an extension with fallback behavior for the core spec?
• How do we restructure the skeleton in the spec?
  • Must we need to separate the skeleton from the node hierarchy? If yes, how do we separate the skeleton?
  • If the skeleton is separated from the node hierarchy, how does animation target a skeleton bone?
  • How do attachments work?
• Others?

Chime in with proposals or other considerations.

[bghgary]

One thought I have is we can possibly keep the skeleton bones still as nodes in the glTF, but these nodes will be referenced only by the skin and never by the scene nodes or its descendants. This has the benefit that animations can still target bones the same way it does right now.

[takahirox]

Two thoughts.

1. Do most of major engines support shared skeleton? If no, done in an extension would be better. In three.js, currently skeleton can be shared in detached mode if we ignore bindMatrix and we separate skeleton from the node hierarchy.

2. If skeleton is shared among some meshes, how can we place another mesh under a bone of one of the meshes? For example, imagine two humanoid meshes share a skeleton and we wanna let the only one humanoid model have a weapon mesh under his/her left hand bone.

[bghgary]

2. If skeleton is shared among some meshes, how can we place another mesh under a bone of one of the meshes? For example, imagine two humanoid meshes share a skeleton and we wanna let the only one humanoid model have a weapon mesh under his/her left hand bone.

This is what I mean by attachments above. You can also attach a different skinned mesh under a bone. In theory, you should be able to attach another node hierarchy to a bone.

[meshula]

For comparison, in USD, the skeleton is stored separately from the joint bindings and weights which are geometry attributes. Skinning is computed given a skeleton and geometry attributes. This allows skeleton and animation sharing, which we considered important for populating crowds and what not. You can see how it's designed here: https://graphics.pixar.com/usd/docs/api/usd_skel_page_front.html

[donmccurdy]

Tagging this issue with "extension" and "glTF next". The current spec allows meshes to share a skeleton, but because the transform is inherited from the skeleton, not the mesh, it probably doesn't enable some of the uses cases — like populating crowds — that developers would want from the feature. For those use cases, refinement in the next version of glTF or an extension in the current version would be necessary. Feedback, additional use cases, or proposals are welcome.