search

ruyo / VRM4U

Runtime VRM loader for UnrealEngine4
Other
1.34k stars 175 forks source link

animation retargeting issue #114

Open chemonoworld opened 2 years ago

chemonoworld commented 2 years ago

Hi! ruyo! Thx for great plugin. I have difficulty in animation retargeting.

Import vrm file to ue4 & ue4 mannequin(skeleton) for vrm is generated well(Import option: create renamed humanoid mesh) Then, I have some problems with animation retargeting. Commonly, It works well for Mixamo animation.

But, The joints are severely distorted like this image below when I try it with live link pose (motion capture plugin: poseAI) image

Sometimes, the problem also happened when I tried it with Mixamo animation. I remember that it happened because of

  1. A-pose/T-pose inconsistency or
  2. ue4 mannequin which is basically included in vrm /ue4 mannequin for (basically included in ue4 project) inconsistency (maybe I didn't choose "Humanoid" in Set up Rig.)

I want to know what this problem stems from? or it can be fixed in unreal engine?

ruyo commented 2 years ago

Hi. The bones of the mannequin and the VRM Skeleton are different in orientation.

If you just want to record the animation, please use ue4mannequin to capture the motion, and retarget it from anim sequence.

If you want to use VRM models in real time, replace the models in the capture tool with VRM models and set it up. Using ue4mannequin as is, you will need to create a dedicated retarget asset. There seem to be several already implemented assets on UE4. You will probably have to use this as a reference to create your own in C++.

I will try to create an asset, but I am not sure if it will work.

image

chemonoworld commented 2 years ago

I'll try it with your guide. Thx!