Write skeleton mesh bones individually?

[GraceEMc]

Hi again, I am wondering if it is possible to export the skeleton as individual bones rather than a single mesh file. If so, could you please direct me to where I might be able to modify the code to do this?

[siddhshenoy]

Hey Grace,

Have you come across a resolution for your requirement? Since I'm interested in doing something similar...

[MarilynKeller]

To export individual meshes, you can use the metadata in StitchedSkeleton:

sk = StitchedSkeleton()

The name and index of each skeleton part: sk.parts = {'Pelvis': 0, 'Spine': 1, 'Spine1': 2, 'Spine2': 3, 'Neck': 4, 'LeftClavicle': 5, 'RightClavicle': 6, 'Sternum': 7, 'Head': 8, 'LeftArm1': 9, 'RightArm1': 10, 'LeftArm2': 11, 'RightArm2': 12, 'LeftHand': 13, 'RightHand': 14, 'LeftLeg1': 15, 'LeftLeg2': 16, 'RightLeg1': 17, 'RightLeg2': 18, 'LeftFoot': 19, 'RightFoot': 20}

The vertices indices of each skeleton part: sk.part2bodyPoints

The faces of each skeleton part: sk.partFaces

Thus, you can export the infered skeleton mesh by individual part. For example to export the humerus,

Let sp be the infered skeleton as here

To export the left humerus as an individual bone mesh:

from gloss_skeleton.gloss.stitched_skel import StitchedSkeleton
sk = StitchedSkeleton()
# The humerus is the part with index 9
hum_verts_id = sk.part2bodyPoints[9]
hum_faces = sk.partFaces[9]

humerus_mesh = Mesh(v = sp.r[hum_verts_id], f = hum_faces)
humerus_mesh.write_ply('infered_humerus.ply')

Let me know if you meet any further issue.

[hwy1992129]

@MarilynKeller I tried... It doesn't work. StitchedSkeleton method doesn't load model.. I added

skel_path = cg.skeleton_model_file
sk.load_model(skel_path)

However, the order of indices has issues.... The result is shown below. I exported all the parts...

[pujades]

Hi, One known issue with Blender is that it can change the vertex order. You have one checkbox in the import /export window to make sure the order of the vertices is preserved. Can you please check if this solves the problem?

[hwy1992129]

Hi, One known issue with Blender is that it can change the vertex order. You have one checkbox in the import /export window to make sure the order of the vertices is preserved. Can you please check if this solves the problem?

I added the code at the end of inference.py. The exported *.ply is shown like the image. It is not related to Blender....

[MarilynKeller]

I see, there seems to be a vertex order issue. Can you share this code in text format? And if you share the resulting part meshes, then I can check them. It is possible that the vertex indices are ok, but not the faces.

[hwy1992129]

def infer_skeleton(skin_pkl_path, skel_mesh_path, skel_pkl_path, skin_mesh_lying_path, gender, display=False, verbose=False):
    """Infer a skeleton mesh given STAR parameters.

    Args:
        skin_pkl_path (str): Path to a pickle file of a dictionary containing STAR parameters ("verts", "betas")
        skel_mesh_path (str): Path to save the infered lying down skeleton mesh
        skel_pkl_path (str): Path to save the infered lying down skeleton object
        skin_mesh_lying_path (_type_): Path to save the corresponding lying down body shape
        gender (str): Gender of the input body mesh, must be in ['male', 'female']
        display (bool, optional): If True, show the meshes during the alignment process. Defaults to False.
        verbose (bool, optional): If True, print the losses during optimization. Defaults to False.
    """

    # Load skin and pose it in lying down pose
    skin_data = pkl.load(open(skin_pkl_path, 'rb'))
    assert set(['verts', 'betas', 'pose']).issubset(skin_data.keys())
    nb_beta = skin_data['betas'].shape[0]
    skin_model = STAR(gender, num_betas=nb_beta)
    skin_model.betas[:] = skin_data['betas']
    pose_lying(skin_model, gender)

    # Visualize
    # if display:
    #     Mesh(skin_model.r, skin_model.f).show() 
    skin_mesh = Mesh(skin_model.r, skin_model.f)

    #Load learned regressors
    reg = Regressors(gender)

    # Infer Skeleton shape in tpose
    skin_betas = skin_data['betas'][:10]
    skeleton_verts = infer_skeleton_shape_verts(skin_betas, reg.betas_regressor, reg.skeleton_pca)

    # Vizualize the infered skeleton shape
    m = Mesh(np.vstack([skin_mesh.v, skeleton_verts]))      

    # load stitched puppet with the regressed skeleton template
    sp = init_stitched_puppet(skeleton_verts)

    # Optimize for the bones location
    optimize_bone_loc(sp, skin_mesh, reg, verbose, display)

    pkl.dump(sp, open(skel_pkl_path, 'wb'))
    Mesh(sp.r, sp.f).write_ply(skel_mesh_path)
    Mesh(skin_model.r, skin_model.f).write_ply(skin_mesh_lying_path)

    sk = StitchedSkeleton()
    skel_path = cg.skeleton_model_file

    sk.load_model(skel_path)

    # Export parts of lying model
    for i in range(21):
        part_verts_id = sk.part2bodyPoints[i]
        part_faces = sk.partFaces[i]

        part_mesh = Mesh(v = sp.r[part_verts_id], f = part_faces)
        part_mesh.write_ply(f'{i}.ply')

    # Template parts of gloss model
    for i in range(21):
        part_faces = sk.partFaces[i]

        part_mesh = Mesh(v = sk.partPoints[i], f = part_faces)
        part_mesh.write_ply(f'temp_{i}.ply')

[MarilynKeller]

If you display the sub-meshes as point clouds only, without the faces, do they look fine?

[hwy1992129]

It is not correct...

[anishdulal]

Hello @MarilynKeller and @hwy1992129, thank you for bringing this into discussion. I also tried the above code given by @MarilynKeller to get humerus but the result is not correct. Have you guys found out the work around for this?

[anishdulal]

Replacing

humerus_mesh = Mesh(v = sp.r[hum_verts_id], f = hum_faces)

by

humerus_mesh = Mesh(v = sk.partPoints[9], f = hum_faces)

gave the desired result. Thank you!

[MarilynKeller]

I added an argument --per_part to save the skeleton mesh per part. You can try it with the last release:

`python main.py --mesh_input data/demo/body_female.ply --gender female -D -v -F --per_part

It exports the different bones as separated mesh files:

[anishdulal]

Hi @MarilynKeller, would it be possible to separate the pelvis as three separate segments (right coxal, left coxal, and sacrum)?

[MarilynKeller]

Hi, I do not have this segmentation, but here is how you can proceed:

1. Export the pelvis mesh using the current "per-part" argument (see readme)
2. Open the mesh in a 3D editing software
3. export the indices list of each subpart you are interested in (right coxal, left coxal, and sacrum)
4. In the OSSO code, before exporting the pelvis mesh, use those lists of vertices to split the mesh into 3 submeshes and export them as 3 different meshes.

For 3., here is how I personnaly proceed.

• Load the pelvis mesh in blender
• go in the script layout of Blender (you can select at the top of the screen)
• copy paste this script in the script subwindow
• In the mesh view subwindow, press tab to switch to edit mode
• press C and select some points of the sacrum
• press CTRL + L to select all the connected vertices
• Run the script by clicking the play button on the script subwindow. This will save the vertices indices as a pickle.

For 4. given a list of vertex indices, you can export a submesh following this example). The trimesh library might also have a native function to do that.

[anishdulal]

Hi @MarilynKeller, thank you so much for your guidance! It worked perfectly!