search

hbertiche / PBNS

108 stars 18 forks source link

Training with custom data #19

Closed SeanHoang2203 closed 8 months ago

SeanHoang2203 commented 1 year ago

Hi,

Thanks for your amazing work and publishing your code. It's really inspiring. I'm having some issues when trying to train with custom body and outfit.

My first question is does this work with other rest pose? Or the rest pose are strictly fixed? I noticed that when I changed the rest pose and re-generate the body mesh using this G, B = smpl.set_params(rest_pose, beta=model._shape, with_body=True) (with model._shape being my own SMPL betas) I obtained a translated model from the original. image

As a workaround, I manually translate my outfit to where the re-generated body is for alignment. I can train PBNS with this (custom rest pose, custom data) but not having good results. So does this relate to the rest pose somehow? For ref, my pose compares to original rest pose image

Another question is how good must the alignment of the outfit and the body be? Is any interpenetration allowed? The same can be asked for the outfit also since my outfit (t-shirt + skirt) is having some interpenetration. image

I hope my questions are clear and thanks again for the work.

hbertiche commented 8 months ago

The rest pose is fixed, but it should be possible to adapt the code to work with arbitrary rest poses. Small interpenetrations in the rest pose like the ones you show are OK.