bharat-b7 / MultiGarmentNetwork

Repo for "Multi-Garment Net: Learning to Dress 3D People from Images, ICCV'19"
282 stars 65 forks source link

Problem while dressing a model. #31

Closed Frank-Dz closed 4 years ago

Frank-Dz commented 4 years ago

Hi~ Thanks for the great work! But here I met a problem. That is when I tried to dress a SMPL model, there was a penetration between SMPL and the pant on the left leg.

image

The method I used to repose the garment is :

def dress(smpl_tgt, body_src, garment, vert_inds, garment_tex = None):
    '''
    :param smpl: SMPL in the output pose
    :param garment: garment mesh in t-pose
    :param body_src: garment body in t-pose
    :param garment_tex: texture file
    :param vert_inds: vertex association b/w smpl and garment
    :return:
    To use texture files, garments must have vt, ft
    '''
    tgt_params = {'pose': np.array(smpl_tgt.pose.r), 'trans': np.array(smpl_tgt.trans.r), 'betas': np.array(smpl_tgt.betas.r), 'gender': 'neutral'}
    smpl_tgt.pose[:] = 0
    body_tgt = Mesh(smpl_tgt.r, smpl_tgt.f)

    ## Re-target
    ret = retarget(garment, body_src, body_tgt)

    ## Re-pose
    ret_posed = pose_garment(ret, vert_inds, tgt_params)
    body_tgt_posed = pose_garment(body_tgt, range(len(body_tgt.v)), tgt_params)

    ## Remove intersections
    ret_posed_interp = remove_interpenetration_fast(ret_posed, body_tgt_posed)
    ret_posed_interp.vt = garment.vt
    ret_posed_interp.ft = garment.ft
    ret_posed_interp.set_texture_image(garment_tex)

    return ret_posed_interp

Seems like the intersections should be removed after adopting this method. Anyone met this problem before? Thanks in advance!

bharat-b7 commented 4 years ago

The code remove_interpenetration_fast removes vertex intersections. What you are seeing in the result is the case where non-vertex points of a face (on the body mesh) intersect with the garment mesh. One way to handle this could be to densely sample points on the body mesh and remove intersections.

Frank-Dz commented 4 years ago

The code remove_interpenetration_fast removes vertex intersections. What you are seeing in the result is the case where non-vertex points of a face (on the body mesh) intersect with the garment mesh. One way to handle this could be to densely sample points on the body mesh and remove intersections.

Thanks!