tr3e / InterGen

[IJCV 2024] InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactions
https://tr3e.github.io/intergen-page/
209 stars 10 forks source link

inference results export to bvh error #42

Open Italian-PAO opened 2 months ago

Italian-PAO commented 2 months ago

Hello, thank you very much for interGen's contribution to this field!

I am using the action results (3d pose) generated by interGen reasoning to redirect some fbx models. My approach is to use the conversion tool provided in momask (https://github.com/EricGuo5513/momask-codes/blob/main/visualization/joints2bvh.py) to convert the 3d pose into a bvh format action, and then use bvh for redirection.

However, I found some problems in the process of converting bvh: When one person in a two-person action is facing forward, the converted BVH action is very correct; but when one person is facing away/sideways, the converted BVH action is very strange, but the visualization of 3D pose is normal (see the two videos in the attachment).

I suspect that there is a problem with the IK solution during the conversion process, but I am not an expert in this area, so I would like to ask you: What might be the problem in this case? If it is really a problem with the IK solution, how to solve this problem?

Maybe this issue is not appropriate to raise under your repo, but I still want to ask for your help, thank you very much!

Looking forward to your reply

大佬你好,非常感谢interGen为这个领域做出的贡献!

我正在用interGen推理生成的动作结果(3d pose)来重定向一些fbx模型,我的做法是使用momask中提供的转换工具 (https://github.com/EricGuo5513/momask-codes/blob/main/visualization/joints2bvh.py) 将3d pose转换成bvh格式的动作,再用bvh来做重定向。

但是在转换bvh的过程中我发现了一些问题: 当双人动作中某一个人的朝向是正对着的时候,转换出的bvh动作是非常正确的;但当某个人是背对/侧对着的时候,转换出的bvh动作就非常奇怪,但3d pose的可视化却是正常的(可以查看附件的两个视频)。

我怀疑是转换的过程中IK解算出现了问题,但我并不是这方面的专家,所以想要请教您一下: 这种情况可能是什么问题?如果真的是IK解算的问题,该如何解决这个问题?

可能这个issue在您的repo下面提并不合适,但还是想请求您的帮助,非常感谢!

期待您的回复

https://github.com/user-attachments/assets/155e7ff3-62ea-4be0-9cf3-1d0e62c65387

https://github.com/user-attachments/assets/6576b3f6-111d-4f88-a78e-ae810fb1567a

tr3e commented 2 months ago

ik优化不鲁棒导致的吧,试下分阶段优化,先优化躯干位置,再优化四肢

Italian-PAO commented 2 months ago

非常感谢大佬的回复!我是这方面的小白,想请教一下大佬有没有可以参考的分阶段优化代码呢?再次感谢!