Issue with skeleton animation

[tito]

I got this model where babylon.js viewer work, but panda3d animation is broken. Dunno what is happening exactly.

Model on babylon.js viewer:

and Panda3D:

benmajid_BIN-modelconverter.zip

[kergalym]

I have same issue with my character, so I still have to use YABEE

[Moguri]

@rdb I know you are busy so I'm just looking for hints if you've got them. This file has seven skins and they all have the skeleton property. For each skin, load_skin() picked node 0 as the root_nodeid. This means, at the end of the day, only the last skin managed to get recorded in self.skeletons. None of the skins have node 0 in its joint list. If I prefer gltf_skin['skeleton'] for the root_nodeid (if present), then this model does appear to load correctly.

So, it appears that we have multiple skins that are ultimately parented to the same root node, but that root node is not part of any of the skins themselves. What do we do if a multiple skins share a root node? Should we try to make use of skin['skeleton'] if available? I know there was talks of getting rid of this property, so we might want to figure out potential issues with our current approach. Do we attempt to combine the skins when there is a collision in self.skeletons?

@tito and @kergalym To avoid this problem, try using a single skeleton/armature if possible. I'd be interested in learning why this file has seven skeletons for one character. Maybe it's an oddity with a particular exporter. Was this exported from Blender?

[Moguri]

Well, my first, dumb attempt at combining the skins didn't work very well (just got a pancake of verts) :D

diff --git a/gltf/converter.py b/gltf/converter.py
index bdf4ff7..5137615 100644
--- a/gltf/converter.py
+++ b/gltf/converter.py
@@ -675,8 +675,10 @@ class Converter():
                 path.pop()

         root_nodeid = common_path[-1]
-
-        self.skeletons[root_nodeid] = skinid
+        if root_nodeid in self.skeletons:
+            self.skeletons[root_nodeid].append(skinid)
+        else:
+            self.skeletons[root_nodeid] = [skinid]

     def load_primitive(self, geom_node, gltf_primitive, gltf_mesh, gltf_data):
         # Build Vertex Format
@@ -958,20 +960,21 @@ class Converter():
         affected_nodeids = set()

         if nodeid in self.skeletons:
-            skinid = self.skeletons[nodeid]
-            gltf_skin = gltf_data['skins'][skinid]
+            skinids = self.skeletons[nodeid]

-            if 'skeleton' in gltf_skin:
-                root_nodeids = [gltf_skin['skeleton']]
-            else:
-                # find a common root node
-                joint_nodes = [gltf_data['nodes'][i] for i in gltf_skin['joints']]
-                child_set = list(itertools.chain(*[node.get('children', []) for node in joint_nodes]))
-                root_nodeids = [nodeid for nodeid in gltf_skin['joints'] if nodeid not in child_set]
-
-            jvtmap.update(self.build_character_joints(char, root_nodeids,
-                                                      affected_nodeids, skinid,
-                                                      gltf_data))
+            for skinid in skinids:
+                gltf_skin = gltf_data['skins'][skinid]
+                if 'skeleton' in gltf_skin:
+                    root_nodeids = [gltf_skin['skeleton']]
+                else:
+                    # find a common root node
+                    joint_nodes = [gltf_data['nodes'][i] for i in gltf_skin['joints']]
+                    child_set = list(itertools.chain(*[node.get('children', []) for node in joint_nodes]))
+                    root_nodeids = [nodeid for nodeid in gltf_skin['joints'] if nodeid not in child_set]
+
+                jvtmap.update(self.build_character_joints(char, root_nodeids,
+                                                          affected_nodeids, skinid,
+                                                          gltf_data))

         cvsmap.update(self.build_character_sliders(char, nodeid, affected_nodeids,
                                                    gltf_data, recurse=recurse))

[kergalym]

@Moguri, my armature is single, but meshes are multiple, may be you wanted to say "use single meshed character"? I think multimeshed character could be an issue.

When I exported my animated character last time I even couldn't activate animation, it looked like it inside an actor but actor.play('anim') did nothing.

[kergalym]

@Moguri I had some tests with it, I think, this issue is a result of non-uniform scaled multimeshed character, something like not 1.0, so, better to join it into single and set scale to 1.0 (otherwise it breaks animation). panda3d-gltf doesn't handle these things well.

[kergalym]

Further tests show me it's more related to non-uniform position and scale (not 0, 0, 0 and not 1.0 respectively) multiple meshes on single character itself doesn't affect.

[Moguri]

@kergalym I'm thinking your issue may be a different one than what @tito is running into. Would you mind filing a separate issue with an example file?

It looks like this asset was created with FBX2gltf, and I think it's doing something really funky with the skins that may be technically legal, but may not map well to the way Panda represents characters. There is not a single "skeleton" as far as I can tell. The seven skins point to different root nodes, but their lists of joints keep overlapping. It also has a couple of single joint skins. No two meshes share a skin. So, you would end up with seven characters each using some joints from the others.

@tito What kind of scene-graph hierarchy would you expect this file to represent? In other words, do you know what was going on with the original FBX file? Was it originally a single character? Are there any special tricks going on?

[rdb]

My first instinct is that we shouldn't try to combine the skins, but respect the skeleton property in case of a conflict. What's less clear to me is in which situations we should ignore or respect the skeleton property.

[rdb]

On second thought, this file has only one animation affecting all the skins. How exactly would this be expected to work, I guess as a subpart animation? Not sure whether Actor could even load such a thing. Maybe combining is the right thing after all, but it seems difficult to pull off.

[tito]

@Moguri i do not know what's going on in the original file. I think i can share it, but i have no idea how the author made the mesh. As a developer/integrator, i just got the model designer sent me, and integrate them into the code :/