A few of custom requests or potential upgrades I hope to see in VRM4U.

[evbahn]

Hi Ruyo! I've been using VRM4U for quite a while on different projects now. It's just amazing! Thank you for keeping it updated! Still trying hard to generate some actual revenue from my projects so I could support your works as well.

I have a few questions I could DM you but I figured there might be developers interested in similar ideas so I'm posting it here.

1. For the Runtime VRM Importer, would you consider adding a function for a 2nd or more VRM model to be imported in runtime? If you find this applicable only for my project and would consider this as a personal request, I'm open to offers!

2. Would it be possible to export the animations along with the VRM that is built-in from the VRoid Studio?

3. This is not part of the VRM4U, I just wanna know by any chance you know how to runtime import an FBX into UE4?

Thanks for your time reading this! Looking forward to your reply!

[ruyo]

Hi. Thanks! If you have any questions, feel free to ask.

1 The runtime load can handle multiple VRM models. This is a bit complicated. A SkeletalMesh is generated at runtime, and you need to keep it in actor. You can apply runtime retargeting to it.

2 I'm not familiar with VRoidStudio, but it will not export the animation. The current VRM file is not supposed to contain animation data. It can be included as a glb, but UniVRM probably doesn't support exporting animations.

3 FBX has no rules for defining PBR materials, front axes, or scales. If you want to use these in your game, you will need your own rules and converters. VRM4U can load FBX at runtime, but this is not practical for the reasons mentioned above.

thanks

[evbahn]

Thanks for the reply!

1 The runtime load can handle multiple VRM models. This is a bit complicated. A SkeletalMesh is generated at runtime, and you need to keep it in actor. You can apply runtime retargeting to it.

I'm putting my priority on this one, currently figuring out how it works by creating multiple same type of characters for testing. Will post questions regarding this matter on this thread! Thanks!

[evbahn]

Hello Ruyo! Sorry for bugging you with questions again.

1. Long ago I saw you have done the finger tracking with the Oculus Quest. https://twitter.com/ruyo_h/status/1356333512505262080?s=20 May I ask for more detail on how to set it up with the VRM4U? (My current VRM4U build is 20210820) Also, will that work in RunTime?

2. Another question is that is it possible to run Control Rig in Runtime? I'm hoping to use the Control Rig after the software is packaged but I wonder if I have to create a whole other system with BP or Widget just to link with the Control Rig instead of using the already wonderfully created Control Rig but currently support only in the Project Viewport.

Thank you!

[ruyo]

Hi. 1. Please refer to this page. It requires source rewrite and build. I have not tested it, but it probably works at runtime. https://ruyo-github-io.translate.goog/VRM4U/05_tracking/?_x_tr_sl=ja&_x_tr_tl=en&_x_tr_hl=ja&_x_tr_pto=nui

2. You will need to modify this ControlRig to work with AnimBP. I have tried to make one as a test. However, the current ControlRig is not very flexible with variables. Also, the linkage with AnimBP is complicated. Therefore, it is difficult to provide it as a general function of a plug-in.

In the future, when the FullbodyIK plugin is officially released, I would like to think about it again.

thanks

[evbahn]

Thanks for the prompt reply!

1. I'll try it and post here regarding whether it works in runtime or not. (I'll probably make a video tutorial about it if I have some free time.)

2. Thanks for keeping up the VRM Control Rig workflow and making it easy for UE4! I'm totally gonna implement it and try to reach its full potential when it releases.

You're awesome! Cheers!

[evbahn]

Hi! I'm back with a question regarding the Hand Tracking.

So I've set the build.cs bUseQuestTracking to true, I reopen the project, open the VRM4U_Tracking level and test the VR Preview, it doesn't seem to work with my Oculus Quest 2 Hand Tracking. In my current project, I could see the Hand Tracking works by adding only the Oculus Hand Component so I assume my Hand Tracking setup is correct. Back in the VRM4U_Tracking level, I could see the VrmModelActor character in the VR but it didn't react or sync with my Oculus at all. Am I missing anything?

[ruyo]

I confirmed that it works with UE4.27.1. Check your Oculus plugin settings, it should work if you only enable the OculusVR plugin.

Hand position tracking works right away. It seems to take a little time to start tracking fingers. (Once it starts, it works smoothly.)

[evbahn]

Hi, I've migrated my project from 4.26 to 4.27 along with VRM4U to the most recent version. I've double-checked my setup to make sure only Oculus VR is the only VR plugin enabled.

But still, I can only see the character with T-Pose.

I've no idea what could alter the hand tracking. Would you mind sharing the hand tracking setup on your Oculus and UE4, please?

[ruyo]

Did you build the plugin? Set bUseQuestTracking to true, and then build it again from VisualStudio.

I think it will work with either UE4.26 or 4.27.

[evbahn]

I didn't build the plugin in VisualStudio! For the life of me, I didn't know that is the main cause for not getting the hand track functioning. I've probably relied on Blueprint for everything too much, plus I'm not a programmer. My bad!! The Hand Tracking works perfectly fine now in VR preview! :D

I'm very sorry that I have one more question about the hand tracking in VRM4U. Currently, my Blueprint Character has quite a lot of custom functions, so maybe it's too late for me to use the BP_VrmModelActor as the parent of my character now. I've added the following component to my character but the hand tracking doesn't seem to work compared to the BP_VrmModelActor.

Do you have any advice for me on what to copy from your BP_VrmModelActor regarding the hand tracking function? If this is too much of a hassle for you I could seek a workaround. Thanks a million!!

[ruyo]

The tracking function is a bit complicated.

HMD and finger tracking is done by BP_HMDTracking. The coordinates calculated by this actor are passed to ABP_VRoidSimple.

If you want to keep the same motion as the character in the sample map, there is a function to copy the pose, try BP_VrmPoseCopy (you can replace the model by overwriting VrmAssetList).

If you want to customize the movement in some way, you can create a new AnimBP with ABP_VRoidSimple as a reference.

[evbahn]

Thanks for the advice!! I've been wrestling with the AnimBP and the Hand Tracking from the plugin and it's still back down on me. It's indeed complicated. I got the fingers tracking but it's very wonky. The fingers joints are twisted weirdly and don't look as smooth as the BP_VrmModelActor. I have to use the workaround using the default Oculus hands for now. I'll go back and see if it works with my character again while it goes. Thanks again!