Unable to convert imported Animation from .fbx to .vmd (keyframes removed for limbs bones)

[zeromizkun]

So I've cleaned up an MMD model, exported it as FBX into other software and created some animations for it (more like retargetted), then I've put it back in blender and tried to export it as vmd but in MMD only the hips, chests and neck are moving, all the main bones connected to the root (legs and arms are not as they have no frames at all after exporting as vmd all frames for all the limb bones are erased). I've tested then doing animations directly in blender and then exporting as FBX, those worked perfectly, then I exported that animations as FBX from blender and imported back then to blender again, then I tried to export it as vmd, but I got the same issue only the hips, chest bones moving.)

Is this a known issue? any way to be able to export an animation that was imported into blender as FBX to VMD?

So everything made short: Making an animation directly in blender and exporting it as vmd works, all bones animated BUT using an imported animation (even the same one that previously worked directly) from .fbx to .vmd does not work, only some bones are animated while others are missing all their keyframes (limbs usually)

I've attached a file containing the original blender file with animation (exporting to vmd directly from there works), the fbx in question from which I would like to import the animation to vmd for all bones (Just simulating as if I've exporting from my retargetting software) and the mmd model (some bones have been given different parents, but issue is still the same with original models)

Thank you! :) really needing help! Bought a motion capture suit just for this.. hope someone can help us! Been fuking my brainz for days haha some tips would be much appreciated!

here is the download for the files: https://mega.nz/#!sBgRzQ7J!bwKtpUKYIaf0c7vCQKD6iwk7ChShjtbulWXrWhOYgT0

[powroupi]

Just tested it, I think it is an issue of name mapping. The imported .fbx model doesn't contain original MMD bone names, so Blender bone names will be used to export to .vmd. You might try change VMD import option Bone Mapper from PMX to Blender when you import the FBX's VMD to PMX model, and be aware of the Scale. :)

Steps which work for me:

1. Import MMDTesting.fbx (default import options).
2. Select FBX's armature object Elven Base ShapeChange_arm, export VMD to test.vmd (default export options).
3. Import Elven Base_ShapeChange.pmx (change import option Scale to 0.08).
4. Import test.vmd to the PMX model (change import option Bone Mapper to Blender, and Margin to 0)
5. Mute IK constraints of PMX model manually, check the animation which should work as expected.

[zeromizkun]

@powroupi Oh Man! That actually work! damn tricky sht haha, thank you so much! Could you answer me one question tho? as I've said I've done changes to the bones of the mmd model, i've replaced the root with the actual hips, I've already unlcocked it's movement I can move the character up and down in MMD and in blender it plays well too, but in miku miku dance the new root (the hips 下半身 ) even tho are unlocked, it gets no movement keyframes only rotation, is it something to do with the way the plugin exports the vmd? Here is a comparison : In blender: https://gyazo.com/6118b3e4085080436ca08e775e1968ea In MMD: https://gyazo.com/6118b3e4085080436ca08e775e1968ea if its nothing to do with the addon then its fine Will see how to fix xd already spent 3 days on the previous issue haha

I've also tried moving all location frames from the hips back to the default root, same as before, works well in blender, no movement, only rotation, in mmd imma be honest, im new but not that new xD thought moving the ketframes to the actual root bone would work but nop..

Thanks!

[Hogarth-MMD]

I'm not sure if this is your problem, but the correct scale of the VMD animation is essential to getting the correct scale of the movement of bones. The scale/size of MMD models is about 10 times greater than most other 3D models. So maybe you are getting movement of your model's bones, but the size of the movement is not great enough for you to see it.

[zeromizkun]

I'm not sure if this is your problem, but the correct scale of the VMD animation is essential to getting the correct scale of the movement of bones. The scale/size of MMD models is about 10 times greater than most other 3D models. So maybe you are getting movement of your model's bones, but the size of the movement is not great enough for you to see it.

Yeah earlier i just remembered powroupi mentioning about scale, so then I went and created an animation by hand in blender with an mmd model with the default scale of 1 (not 0.08) and then imported it into mmd and the hips move, so now just gotta find the right export scale to work right and theeen finally done! ;-; time to make my anime series xD anybody knows any good auto lipsync solution for mmd? besides LipSynchloid (which btw doesn't do too bad either, cleaning is a pain but eh its something)

Thank you both btw! ♥ You have seriously no idea for how long we've been dreaming about this (my wife and I) :3 Cheers!

[powroupi]

@zeromizkun you might try the following steps if you want to use the .vmd of FBX model on MMD. :)

1. Import MMDTesting.fbx (default import options).
2. Select FBX's armature object Elven Base ShapeChange_arm, export VMD to test_mmd.vmd (change export option Scale to 12.5).
3. Import Elven Base_ShapeChange.pmx (default import options (Scale 1.0)).
4. Import test_mmd.vmd to the PMX model (change import option Bone Mapper to Blender, and Margin to 0)
5. Mute IK constraints of PMX model manually, check the animation which should work as well.
6. Load Elven Base_ShapeChange.pmx and test_mmd.vmd on MMD.
7. Disable IK bones manually at first frame, check the animation which should work now.

[zeromizkun]

Ey! Here it is! https://gyazo.com/8d6c212b6d61f0f292534cf03f315b54 Finally haha! Yesh thank you @powroupi I followed how ya said the first time, just that when I export the vmd I set the scale to 12 👍 and now it works ! Jesus! YOU GENIUS! Thank you! ;-;

[powroupi]

Correct above steps:

6. Re-export VMD of the PMX model to test_mmd_1.vmd (default import options (Scale 1.0)) to use correct bone names for MMD.
7. Load Elven Base_ShapeChange.pmx and test_mmd_1.vmd on MMD.
8. Disable IK bones manually at first frame, check the animation which should work now.

[powroupi]

@zeromizkun yeah, just be aware of bone names and Scale, I guess you understand it now. :smile:

[Hogarth-MMD]

anybody knows any good auto lipsync solution for mmd? besides LipSynchloid (which btw doesn't do too bad either, cleaning is a pain but eh its something)

https://github.com/powroupi/blender_mmd_tools/issues/90

This solution (Julius free speech recognition and a Blender python add-on) is VERY WIP. The quality of lip sync animation is not very good. I have a .pz2 morphs exporter Blender add-on which can be used with DAZ Mimic and DAZ Victoria 4. It isn't perfect though. I haven't uploaded it for other people because DAZ Mimic and DAZ Victoria 4 are commercial products, and I don't want people blaming me if they spend money, and then my add-on does not work as well as they would like.

[Hogarth-MMD]

The only other lip sync/facial animation solution that I know of would be facial motion capture. I don't know what hardware or software you would need to buy for that, but the models that you are working with would need to be rigged with facial bones and facial vertex groups.

[Hogarth-MMD]

I should also mention Vocaloid. When you make a song in Vocaloid, you can export a VMD lips animation for that song. The free version of Vocaloid is called Utau, but I don't know if Utau also has that feature.

[Hogarth-MMD]

A talking animation is actually quite complex. Even if you can automatically create a lip sync and eyes blinking animation, more work still needs to be done. In a complete talking animation, there are facial expressions, facial gestures, neck movement, arms movement, maybe whole body movement.

[zeromizkun]

@Hogarth-MMD Ay thank you so much for the info! :D I know all I need for facial mocap but for mmd I think just manual and LipSynchloid as no realistic mimic is needed haha :D, for body mocap I use xsens awinda and retargetting is done in Webanimate, and now Blender! All those together make amazing sht haha! Now getting to work! You will hear of us! And all thanks to ya guys :) Cheers!

[Hogarth-MMD]

The Shrek movies have some of the best facial animation that I have seen in 3D animation. I wish that Dreamworks would sell to me the models and animations from Shrek for a nice cheap price, but of course this is wishful thinking. :smile:

[Hhieumeo]

Ey! Here it is! https://gyazo.com/8d6c212b6d61f0f292534cf03f315b54 Finally haha! Yesh thank you @powroupi I followed how ya said the first time, just that when I export the vmd I set the scale to 12 👍 and now it works ! Jesus! YOU GENIUS! Thank you! ;-;

OMG can you make a video on how to do that? I'm new to Blender and cannot figure it out 🥲 I tried many ways to export fbx animation to vmd but nothing works for me :(((