search

anindita127 / IMoS

"IMoS: Intent-Driven Fullbody Motion Synthesis for Human-Object Interaction". Proceeding of EUROGRAPHICS 2023.
MIT License
82 stars 9 forks source link

Knife unnatural predictions #7

Closed braunjon closed 1 year ago

braunjon commented 1 year ago

Hi,

I am running the test_synthesis.py with the pretrained checkpoints. For all 3 knife motions (S10) I get a very unnatural output: image

Do you also get this, or is something wrong with my setup?

Thanks for checking.

anindita127 commented 1 year ago

Hi,

Thanks for your interest in our work. Is this sort of unnatural prediction only happening for the knife? Can you walk me through the steps you followed?

braunjon commented 1 year ago

I have looked at most motions but not at all and so far I have only seen it for the knife.

I just followed the README.MD Getting started. I use the decimated meshes like described here: #1. After the setup I run python src/test/test_synthesis.py. And then in the render_smplx.py I adjust the dir_path='./save/pretrained_models/exp_31_model_Interaction_Prior_Posterior_CVAE_clip_lr_0-0005_batchsize_64_latentD_100_languagemodel_clip_usediscriminator_False_/test/' to the generated data.

(Side Note: In the render_smplx.py when calling SMPLXLayer on line 56-60 I think setting the v_template is missing.)

anindita127 commented 1 year ago

Thanks for pointing it out. It could be that synthesizing movements with the knife falls under a failure case as the knife was a rare object in the training data . More details of such scenarios are mentioned in the limitations section of our paper,