search

YuliangXiu / ECON

[CVPR'23, Highlight] ECON: Explicit Clothed humans Optimized via Normal integration
https://xiuyuliang.cn/econ
Other
1.11k stars 107 forks source link

ECON this job can be used for SCANimate？ #51

Open Bill-WangJiLong opened 1 year ago

Bill-WangJiLong commented 1 year ago

Hello, ECON this job can be used for SCANimate? It looks like their output is the same.

YuliangXiu commented 1 year ago

Yes, you can use either the whole ECON (xxx_full.obj) or the partial ECON (depth map optimized from d-BiNI) as inputs of SCANimate.

Bill-WangJiLong commented 1 year ago

Yes, you can use either the whole ECON (xxx_full.obj) or the partial ECON (depth map optimized from d-BiNI) as inputs of SCANimate.

Thank you for your reply! Can the pose parameters and transl parameters of the smpl model required by SCANimate be obtained directly with this code in infer.py?

if not osp.exists(smpl_obj_path): smpl_obj. export(smpl_obj_path) smpl_info = { "betas": optimal_betas[idx].detach().cpu().unsqueeze(0), "body_pose": rotation_matrix_to_angle_axis(optimed_pose_mat[idx].detach() ).cpu().unsqueeze(0), "global_orient": rotation_matrix_to_angle_axis(optimed_orient_mat[idx].detach() ).cpu().unsqueeze(0), "transl": optimed_trans[idx].detach().cpu(),

In addition, I also want to ask whether it is necessary to remove some faces in full_obj like ICON does? Or do some changes need to be made to SCANimate?

YuliangXiu commented 1 year ago

Yes, you can use either the whole ECON (xxx_full.obj) or the partial ECON (depth map optimized from d-BiNI) as inputs of SCANimate.

Thank you for your reply! Can the pose parameters and transl parameters of the smpl model required by SCANimate be obtained directly with this code in infer.py? if not osp.exists(smpl_obj_path): smpl_obj. export(smpl_obj_path) smpl_info = { "betas": optimal_betas[idx].detach().cpu().unsqueeze(0), "body_pose": rotation_matrix_to_angle_axis(optimed_pose_mat[idx].detach() ).cpu().unsqueeze(0), "global_orient": rotation_matrix_to_angle_axis(optimed_orient_mat[idx].detach() ).cpu().unsqueeze(0), "transl": optimed_trans[idx].detach().cpu(), In addition, I also want to ask whether it is necessary to remove some faces in full_obj like ICON does? Or do some changes need to be made to SCANimate?

Yes, and Yes, just use the visible faces, but you should modify the dataloader of SCANimate a little bit.

Bill-WangJiLong commented 1 year ago

What should be modified for SCANimate? I noticed that the pose parameter is divided into global rotation and body_pose. Do you mean that it needs to be modified here?

ireneisme commented 1 year ago

What should be modified for SCANimate? I noticed that the pose parameter is divided into global rotation and body_pose. Do you mean that it needs to be modified here?

Hello, sir! Have you ever finished this piece of work? I am also confused about how to convert the output of ECON which has fewer body_pose to the corresponding smpl model one.