doubt in generating fbx, gen_dataset script and skin prediction

[remiwu]

Hi, I am trying to run the maya_save_fbx.py, gen_dataset.py, and quick_start.py and I'm encountering some problems as follow:

1.I used maya_save_fbx.py to integrate ori.obj and ori_rit.txt. The bones in the fbx generated are different from the bones in the fbx provided in quick_start file, especially the end orientation of the bones, as shown in the following figure ( take 2586_ori.fbx as an example, the left is provided in quick_start fbx, the right is generated by me ).I don't know what caused this difference and whether it will affect the animation generation.

2. When I run gen_dataset.py(I have downloaded the pre-processed data and changed the root_folder), I got some errors: Traceback (most recent call last): File "c:/Users/Remi/Desktop/git_project/RigNet/gen_dataset.py", line 168, in <module> genDataset(0) File "c:/Users/Remi/Desktop/git_project/RigNet/gen_dataset.py", line 122, in genDataset skin = rig_info.joint_skin[vert_remesh_id] IndexError: list index out of range

3.The skinning effect generated by my own model is not very good. For example, when I rotate the base of the left leg, many unrelated vertices are moved, or when I rotate the right arm, no vertices are moved.

How can I fix these? Thank you~

[zhan-xu]

These are all hard questions.

1. Are you using Maya or Blender? It seems the default local frame orientation is not set correctly?
2. Could you check whether you load the *_info.txt successfully, i.e., is rig_info.joint_skin empty?
3. Have you observed any part of the bones falling outside the mesh? This will lead the volumetric geodesic calculation wrong. Another possibility is if the mesh is watertight. Still, From the current results, maybe the vertices order is messed up. You can send this model to me and I can try from my end to see whether it is a bug or supposed to be a failure case.

[remiwu]

Thank you for your reply!

1. I used maya to run maya_save_fbx.py and blender to display the fbx because I am more familiar with blender. But the default local frame orientation both select "Automatic Bone Orientation" when import the tow into blender.
2. The rig_info.joint_skin is not empty but less then remesh_obj_v.shape[0].
3. I have sent it to your email zhanxu@cs.umass.edu, thanks for your help.

Looking forward to your reply. Thank you very much!

[zhan-xu]

1. Is it also wrong in Maya?
2. Did you use "rig_info_remesh" and "obj_remesh"? See the picture below.

[zhan-xu]

are you using open3d==0.9.0?

[remiwu]

Thanks for your help! After I installed the correct version of open3d, the second and third problem was solved . However, when I used the original obj as _ori.obj file, which wasn't remeshed, the effect of skinning is not so good as _remeshed.obj. As for the first problem, when I imported the fbx to maya, there were always a lot of messy lines. And I haven't found the way to see the rig of fbx. T^T

[zhan-xu]

Hi, our networks are trained with remeshed objs, and work for models with 1K-5K vertices. We then immigrate the predicted skinning weights to the original mesh based on nearest neighbor. I would not suggest directly use the original mesh with arbitrary number of vertice. The wires are joints. You can adjust "joint size" in Maya.