Closed GraceEMc closed 7 months ago
Hey Grace,
Have you come across a resolution for your requirement? Since I'm interested in doing something similar...
To export individual meshes, you can use the metadata in StitchedSkeleton:
sk = StitchedSkeleton()
The name and index of each skeleton part:
sk.parts = {'Pelvis': 0, 'Spine': 1, 'Spine1': 2, 'Spine2': 3, 'Neck': 4, 'LeftClavicle': 5, 'RightClavicle': 6, 'Sternum': 7, 'Head': 8, 'LeftArm1': 9, 'RightArm1': 10, 'LeftArm2': 11, 'RightArm2': 12, 'LeftHand': 13, 'RightHand': 14, 'LeftLeg1': 15, 'LeftLeg2': 16, 'RightLeg1': 17, 'RightLeg2': 18, 'LeftFoot': 19, 'RightFoot': 20}
The vertices indices of each skeleton part:
sk.part2bodyPoints
The faces of each skeleton part:
sk.partFaces
Thus, you can export the infered skeleton mesh by individual part. For example to export the humerus,
Let sp
be the infered skeleton as here
To export the left humerus as an individual bone mesh:
from gloss_skeleton.gloss.stitched_skel import StitchedSkeleton
sk = StitchedSkeleton()
# The humerus is the part with index 9
hum_verts_id = sk.part2bodyPoints[9]
hum_faces = sk.partFaces[9]
humerus_mesh = Mesh(v = sp.r[hum_verts_id], f = hum_faces)
humerus_mesh.write_ply('infered_humerus.ply')
Let me know if you meet any further issue.
@MarilynKeller I tried... It doesn't work. StitchedSkeleton method doesn't load model.. I added
skel_path = cg.skeleton_model_file
sk.load_model(skel_path)
However, the order of indices has issues.... The result is shown below. I exported all the parts...
Hi, One known issue with Blender is that it can change the vertex order. You have one checkbox in the import /export window to make sure the order of the vertices is preserved. Can you please check if this solves the problem?
Hi, One known issue with Blender is that it can change the vertex order. You have one checkbox in the import /export window to make sure the order of the vertices is preserved. Can you please check if this solves the problem?
I added the code at the end of inference.py. The exported *.ply is shown like the image. It is not related to Blender....
I see, there seems to be a vertex order issue. Can you share this code in text format? And if you share the resulting part meshes, then I can check them. It is possible that the vertex indices are ok, but not the faces.
def infer_skeleton(skin_pkl_path, skel_mesh_path, skel_pkl_path, skin_mesh_lying_path, gender, display=False, verbose=False):
"""Infer a skeleton mesh given STAR parameters.
Args:
skin_pkl_path (str): Path to a pickle file of a dictionary containing STAR parameters ("verts", "betas")
skel_mesh_path (str): Path to save the infered lying down skeleton mesh
skel_pkl_path (str): Path to save the infered lying down skeleton object
skin_mesh_lying_path (_type_): Path to save the corresponding lying down body shape
gender (str): Gender of the input body mesh, must be in ['male', 'female']
display (bool, optional): If True, show the meshes during the alignment process. Defaults to False.
verbose (bool, optional): If True, print the losses during optimization. Defaults to False.
"""
# Load skin and pose it in lying down pose
skin_data = pkl.load(open(skin_pkl_path, 'rb'))
assert set(['verts', 'betas', 'pose']).issubset(skin_data.keys())
nb_beta = skin_data['betas'].shape[0]
skin_model = STAR(gender, num_betas=nb_beta)
skin_model.betas[:] = skin_data['betas']
pose_lying(skin_model, gender)
# Visualize
# if display:
# Mesh(skin_model.r, skin_model.f).show()
skin_mesh = Mesh(skin_model.r, skin_model.f)
#Load learned regressors
reg = Regressors(gender)
# Infer Skeleton shape in tpose
skin_betas = skin_data['betas'][:10]
skeleton_verts = infer_skeleton_shape_verts(skin_betas, reg.betas_regressor, reg.skeleton_pca)
# Vizualize the infered skeleton shape
m = Mesh(np.vstack([skin_mesh.v, skeleton_verts]))
# load stitched puppet with the regressed skeleton template
sp = init_stitched_puppet(skeleton_verts)
# Optimize for the bones location
optimize_bone_loc(sp, skin_mesh, reg, verbose, display)
pkl.dump(sp, open(skel_pkl_path, 'wb'))
Mesh(sp.r, sp.f).write_ply(skel_mesh_path)
Mesh(skin_model.r, skin_model.f).write_ply(skin_mesh_lying_path)
sk = StitchedSkeleton()
skel_path = cg.skeleton_model_file
sk.load_model(skel_path)
# Export parts of lying model
for i in range(21):
part_verts_id = sk.part2bodyPoints[i]
part_faces = sk.partFaces[i]
part_mesh = Mesh(v = sp.r[part_verts_id], f = part_faces)
part_mesh.write_ply(f'{i}.ply')
# Template parts of gloss model
for i in range(21):
part_faces = sk.partFaces[i]
part_mesh = Mesh(v = sk.partPoints[i], f = part_faces)
part_mesh.write_ply(f'temp_{i}.ply')
If you display the sub-meshes as point clouds only, without the faces, do they look fine?
It is not correct...
Hello @MarilynKeller and @hwy1992129, thank you for bringing this into discussion. I also tried the above code given by @MarilynKeller to get humerus but the result is not correct. Have you guys found out the work around for this?
Replacing
humerus_mesh = Mesh(v = sp.r[hum_verts_id], f = hum_faces)
by
humerus_mesh = Mesh(v = sk.partPoints[9], f = hum_faces)
gave the desired result. Thank you!
I added an argument --per_part to save the skeleton mesh per part. You can try it with the last release:
`python main.py --mesh_input data/demo/body_female.ply --gender female -D -v -F --per_part
It exports the different bones as separated mesh files:
Hi @MarilynKeller, would it be possible to separate the pelvis as three separate segments (right coxal, left coxal, and sacrum)?
Hi, I do not have this segmentation, but here is how you can proceed:
For 3., here is how I personnaly proceed.
For 4. given a list of vertex indices, you can export a submesh following this example). The trimesh library might also have a native function to do that.
Hi @MarilynKeller, thank you so much for your guidance! It worked perfectly!
Hi again, I am wondering if it is possible to export the skeleton as individual bones rather than a single mesh file. If so, could you please direct me to where I might be able to modify the code to do this?