Split out the structural info of Skeleton3D into a Resource class

[MajorMcDoom]

Describe the project you are working on

An indie VR game.

Describe the problem or limitation you are having in your project

At runtime and at edit time (for tools), I need to be able to unparent some bones for the purposes of procedural, physics-driven animation. I also need to unparent some bones for character customization because I'm using the same model for different characters (e.g. scaling some segments of the body for different body shapes). This unparenting cannot be done in Blender, because the authoring of the model requires the parentage for tweaking / weight-painting, IK previews, etc.

Godot's current skeleton system is presents numerous challenges because the entirety of skeletal data, whether it's state or structure, lies within a Node. The structural information about a skeleton, like its hierarchical composition, its rest pose, its bone count, bone names, etc. are all treated as mutable properties of the node, instead of as a reusable Resource.

In practice, this means that there is no "underlying source" for what a skeleton's structure is. My tool scripts have to remember the original structure of a skeleton in order to revert back to them. However, this "memory" also has to gracefully survive reimports of the underlying model that might have changed the structure.

I also need to nest things underneath my skeletons, like bone attachments, facial features, etc, but for tech reasons, I can't have my characters be simply inherited scenes of my imported models. This means I have to keep an imported model within my character scenes as a child node, with Editable Children turned on. This makes it extremely easy for a reimport to mess up references, because the entire scene gets reimported, not just a resource like a mesh or a material. I cannot simply make my own Skeleton3D and MeshInstance3D, and have them reference imported resources because the skeleton's purely-node nature is preventing such a workflow.

I'm also unable to reassign skeletons to a Skeleton3D node without writing a bunch of code. For example if my Skeleton3D currently has a penguin skeleton, and I want it to instead have a polar bear skeleton, I must either write code to copy skeletal data from another Skeleton3D node (that either has to be instantiated on-demand, or always exists instantiated in memory somewhere), or I have to replace my current Skeleton3D with a fresh instance, meaning my skeleton cannot be a stable reference. And as mentioned previously, it cannot even be the root of an instantiated scene - I have to dig into its hierarchy to find it. It also means I have to reinstantiate the MeshInstance3D too.

This is tedious, but it is also inconsistent design-wise from the Resource-Node relationships in the rest of the engine. I believe Skeleton3D is the only node in the engine that cannot swap out its "contents" without relying on another Skeleton3D. It also means that while the user is able to manually create a Skeleton3D node, there are almost no cases where it would be practical to do so, as the current system makes it impractical to use without imported scenes. Blend shapes and material overrides are a perfect example of how a node provides runtime values that plug into a structure provided by a reference resource.

NOTE: I am aware of SkeletonProfile, BoneMap and retargeting. And although there is overlap in terminology here, the similarities are superficial, and the use cases are very different.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

I propose to rename Skeleton3D to SkeletonInstance3D, and create a new resource type called Skeleton3D that contains all the structural info of a skeleton. It would be a reusable and saveable resource that isn't a scene. All the stateful information of a skeleton would live in SkeletonInstance3D.

For my current use cases, it means:

• My scripts would be able to freely modify and unparent bones in a SkeletonInstance3D, but also be able to easily revert back to its referenced Skeleton3D resource.
• Reimports would be non-destructive, because my skeleton is my own node, not in an imported child scene. All my custom children like facial features, bone attachments, etc. would remain interact, and all node references would remain intact.
• Even reimports at runtime would successfully sync to the game, allowing for faster iteration.
• I would be able to reuse the same node (maintaining references) and swap out its skeleton. Can even do it via the editor with a couple clicks.

(An alternative would be to keep the node name as Skeleton3D and make a new resource type called SkeletonStructure3D or something like that. The first suggested naming scheme would be more consistent with the naming of the mesh classes, but it might be confusing for current users.)

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

The SkeletonStructure3D would contain the following information that currently lives in Skeleton3D:

• bone count
• bone names
• bone parents
• rest poses

Each Skeleton3D would then have a member structure of type SkeletonStructure3D, and would only contain the remaining stuff that only pertains to runtime state:

• current bone poses
• current bone parents
• pose overrides

Example usage:

@export var si: SkeletonInstance3D

func _ready():

    si.set_bone_parent(2, -1)
    si.reset_hierarchy()
    si.skeleton = load("res://characters/polar_bear.skel")

This also has the advantage of keeping the API for SkeletonInstance3D a lot cleaner and centered around the more common uses for programmers. i.e. it is a lot more common for people to be dealing with the runtime state of a skeleton, rather than modifying its rest poses and bone names. Those things would be hidden away in Skeleton3D, which can still be accessed for advanced use cases, such as severing limbs, etc.

If this enhancement will not be used often, can it be worked around with a few lines of script?

No, it requires quite a bit of custom tooling to work around.

Is there a reason why this should be core and not an add-on in the asset library?

This concerns the architecture of core nodes and resources.

[DigitalN8m4r3]

interesting, a lot of potential here... An example project might help further evaluation. anyways, i got no say here but am lookin forward how this plays out +1

[fire]

Notes

This proposal suggest we do a fake skeleton resource in a node that is then inserted into a graphics side Godot skeleton server system.

Seems like a lot of work so we'll need to figure out what exactly is wanted, optimization priorities are competing with workflow improvements.

There's also compatibility breakage, so this Skeleton3D node with a Skeleton resource might be renamed.

[MajorMcDoom]

@fire and I had a big conversation back and forth about this, and it definitely seems like I need to consolidate my pitch here, because it's apparently got a lot of distracting details.

I need to be able to:

• Author my character without fearing that my node/tech setup will be destroyed every time I reimport a glb model.
• Reset a Skeleton3D to its imported state (parentage, bone names, rest poses).

Whatever solution can achieve those goals is fine.

[MajorMcDoom]

Possible alternatives:

• Add revert button(s) for Skeleton3D inspector. It's there for every other property, this should be no exception.
• Make a model import option to put the Skeleton3D at the very top level.

[fire]

Add revert button(s) for Skeleton3D inspector. It's there for every other property, this should be no exception.

Make a model import option to put the Skeleton3D at the very top level.

I think adding these two options is doable. Which one do you want first?

[MajorMcDoom]

I think adding these two options is doable. Which one do you want first?

I think the top-level skeleton, if possible. But we might get more mileage out of it if it were just an option in the importer to select which node should be the root. Right now it puts the entire scene under a new node, and only lets you choose its type.

[fire]

https://github.com/godotengine/godot/pull/88824#issuecomment-1964231364 has a proposal for exporting a skeleton as a SkeletonProfile.

I don't think there's a way to turn a SkeletonProfile into a scenetree, but it's theoretically possible.

The other part of this is being able to replace a skeleton with a SkeletonProfile which you can use to approximate the "revert" Skeleton3D feature.

[MajorMcDoom]

godotengine/godot#88824 (comment) has a proposal for exporting a skeleton as a SkeletonProfile. I don't think there's a way to turn a SkeletonProfile into a scenetree, but it's theoretically possible.

What I mean is a way to choose any of the nodes that are in the imported gltf to act as the root of the Godot PackedScene.

The other part of this is being able to replace a skeleton with a SkeletonProfile which you can use to approximate the "revert" Skeleton3D feature.

I am already approximating the revert feature using a tool script that reads from the imported scene and copies over data to a skeleton. But revert needs to be core. Any property on any node in an instantiated scene or resource needs a revert button in the editor. If you just let the user manually edit arrays instead of having a bone tree editor, those arrays would all be revertable, so this is just a UI problem.

[lyuma]

Interesting that there is no revert icon on those properties... the entire skeleton inspector is kind of done as a hack, rather than skinning individual properties.... and it's like this for a very good reason: because the inspector is using a Tree to hold the bones and does not want duplicated inspector fields for every bone in the skeleton.

@MajorMcDoom Would it be possible to find or open an issue about the lack of revert on the individual skeleton bones? That is a bug. Note also that most imported files are in their rest pose by design. And the Skeleton3D menu has an option to reset all poses to rest. But also, it could be useful to have a bulk "revert all", so I do see the appeal.

I'm using the same model for different characters (e.g. scaling some segments of the body for different body shapes). This unparenting cannot be done in Blender, because the authoring of the model requires the parentage for tweaking / weight-painting, IK previews, etc.

I would suggest taking a look at my Skeleton Merge Tool addon: https://github.com/V-Sekai/godot-skeleton-merge-tool It might offer another approach to fitting different versions of a character, actually making use of scene composition by appending bones to the base skeleton and directing the "skeleton" NodePath of another skeleton's mesh at the root skeleton.

Auto-parenting one skeleton (containing an outfit, for example) to another Including options to mirror and move individual bones (Lock Bone) I'd love to see your feedback on this.

[lyuma]

Also, I would like to offer a rebuttal to the need for another resource type. What if this could be built on top of the existing SkeletonProfile resource type for this purpose. It contains precisely the data you are asking for:

• bone count
• bone names
• bone parents
• rest poses As well as a little bit of extra optional metadata about a graphical representation for the inspector and the direction the bone is intended to face (such as towards a child)

True, the extra metadata might not be needed, but I don't think it warrants an entirely new resource type.

[MajorMcDoom]

Interesting that there is no revert icon on those properties... the entire skeleton inspector is kind of done as a hack, rather than skinning individual properties.... and it's like this for a very good reason: because the inspector is using a Tree to hold the bones and does not want duplicated inspector fields for every bone in the skeleton.

@MajorMcDoom Would it be possible to find or open an issue about the lack of revert on the individual skeleton bones? That is a bug. Note also that most imported files are in their rest pose by design. And the Skeleton3D menu has an option to reset all poses to rest. But also, it could be useful to have a bulk "revert all", so I do see the appeal.

I'll settle for a revert all button. Note that I'm talking about the one that usually has this icon: ↪️ It just reverts to whatever the underlying resource contains. In this case, it would be the PackedScene. I'm not talking about reverting to rest pose.

I would suggest taking a look at my Skeleton Merge Tool addon: https://github.com/V-Sekai/godot-skeleton-merge-tool It might offer another approach to fitting different versions of a character, actually making use of scene composition by appending bones to the base skeleton and directing the "skeleton" NodePath of another skeleton's mesh at the root skeleton.

I'd love to see your feedback on this.

Your approach is very interesting but it is more complexity than I require. My characters are penguins. You can see here all the characters made from the same mesh and skeleton.

I also need to unparent the bones because the chest can expand and contract to breathe, and the limbs all need to be able to stretch. It's a stylistic thing.

[MajorMcDoom]

Also, I would like to offer a rebuttal to the need for another resource type. What if this could be built on top of the existing SkeletonProfile resource type for this purpose. It contains precisely the data you are asking for:

• bone count
• bone names
• bone parents
• rest poses As well as a little bit of extra optional metadata about a graphical representation for the inspector and the direction the bone is intended to face (such as towards a child)

True, the extra metadata might not be needed, but I don't think it warrants an entirely new resource type.

Totally fair. I think I've already moved past the need for an extra resource type. Hence why the big reframing a couple posts back with the two main problems. Any solution would be great. I might just turn this into other bug reports and proposals. This one is way too big and loaded right now.

[fire]

I don't think there's a way to turn a SkeletonProfile into a scenetree, but it's theoretically possible.

Would you use that feature?

[MajorMcDoom]

I don't think there's a way to turn a SkeletonProfile into a scenetree, but it's theoretically possible.

Would you use that feature?

Can you describe how this would solve my issues? You've gone back to this one a few times now and I don't see it.

[fire]

Hypothetical workflow

Make a model import option to put the Skeleton3D at the very top level.

1. You use the feature Retargeting option to use a template for silhouette to export a specific Skeleton3D as a SkeletonProfile. This covers option in the importer to select which node should be the root.
2. You convert SkeletonProfile to a PackedScene (Like a Skeleton3D at the top level). Make a model import option to put the Skeleton3D at the very top level. This is missing BoneAttachments and meshes though.
3. Use in your workflow.

[MajorMcDoom]

Hypothetical workflow

Make a model import option to put the Skeleton3D at the very top level.

1. You use the feature Retargeting option to use a template for silhouette to export a specific Skeleton3D as a SkeletonProfile. This covers option in the importer to select which node should be the root.
2. You convert SkeletonProfile to a PackedScene (Like a Skeleton3D at the top level). Make a model import option to put the Skeleton3D at the very top level. This is missing BoneAttachments and meshes though.
3. Use in your workflow.

This is no different from me making a new scene out of a skeleton duplicated from my imported scene. The connection is severed so it won't refresh on reimport. This approach is actually more work because of a redundant conversion into a profile and back.

[MajorMcDoom]

I'm going to close this proposal, as it is big and loaded. I have filed a bug report for the Skeleton3D, and will consider opening another proposal for more flexible mesh import options. Thanks y'all.