VPD pose import-export? (& BVH to VMD)

[Hogarth-MMD]

Hello @powroupi! How are you doing? I congratulate and thank you for all of your excellent work on MMD tools. Can you please add support for the import and export of .vpd (Vocaloid Pose Data) files? MMD tools already has support for import and export of .vmd files, so this should not be too difficult a task should it?

[Hogarth-MMD]

Hi @ptrthomas . Yes, I did try that, and I congratulate you very much on your excellent work. I have started a new issue about Japanese to English translation here: https://github.com/powroupi/blender_mmd_tools/issues/61

[Hogarth-MMD]

@powroupi Thank you for the link to the BVH to VMD converter, but this command line program did not work for me. I got this message, but no VMD file: BVH?????????????

[powroupi]

@Hogarth-MMD I'm not sure why it doesn't work for you, I'm sorry. :disappointed: What is the output if you enter command "java -version"? (it requires Java JRE 1.5+)

The manual is here: http://atupdate.web.fc2.com/manual.htm Convert example:

• b-momi.bat -bvh tennis.bvh -map boneMap.txt (output "tennis.vmd")

Make bone map example:

• b-momi.bat -bvh tennis.bvh -map boneMap.txt -makeMap (output "boneMap.txt")

[MarieRose1301]

In LiveAnimation, it's possible to import all bone rotations, including fingers and whatever(for game rip animations) however that "bvh extension" plugin is broken as it doesn't import bone translations Is it possible to somehow get all bone(not just root bone) translations in mmd?

[Hogarth-MMD]

In English, the word translation can mean changing location or it can mean translation from one language to another. In an MMD character there are different types of bones, most of which can rotate, but not translate (change location). If the character has been rigged correctly, generally, the root bone, center bone and IK bones are enabled for translation.

[MarieRose1301]

i know that by translation i meant the bone movement and you don't get what i said, i know the bone types very well. for instance, i have an animation from a game which uses facial bones which have xyz rotation and xyz translation channels. live animation can only read the rotations for all of those bones but it ignores the bone movements completely like on that pic. the mouth bones are supposed to move, but because of the program's inability to read the xyz translations the lips look wrong. the only workaround i've found so far is copying animation curves in 3ds max to a random bvh model and then exporting that animation as a root bone translation, but it messes up the scale greatly and takes too much work as there are 48 bones to do that for

[Hogarth-MMD]

@MarieRose1301 If you want me to help you in the best way, maybe you need to upload the .blend file for me to look at it. Your problem is not completely clear to me, but maybe the bone types of the facial bones need to be edited in PMD editor or PMX editor. And maybe you need to choose the correct export scale when you export a VMD file from Blender.

[MarieRose1301]

@Hogarth-MMD Here, the model and blend file. The model's scale factor was 2.5 but I used a different pmx exporter http://sta.sh/02521x54fng

[Hogarth-MMD]

@MarieRose1301 Facial bones cannot be animated by rotations alone. If the LiveAnimation software cannot translate these bones, then you need to use different software.

[Hogarth-MMD]

@MarieRose1301 Okay, I downloaded this file, and I will take a look at it.

[MarieRose1301]

@Hogarth-MMD i've just tried blender mmd tools to export the vmd file and for some reason it exported a blank vmd file i might be doing something wrong

[MarieRose1301]

actually nope :/ it's become weird when i used blender tools

[Hogarth-MMD]

@MarieRose1301 The problem which you are having seems to be caused by the fact that the scale of your BVH animation is VERY MUCH smaller than the scale of your PMX model. You need to scale your animation to X 34 of its original size. Then I think that it should work OK.

[MarieRose1301]

@Hogarth-MMD I'm sorry for being annoying but I really don't get anything... I've scaled it on export and got this. Do you need a raw imported .tmc and .tdpack as .blend file before i've shortened bone names and exported the model? By the way the blender tools won't export the pmx file for some reason :/

[Hogarth-MMD]

@MarieRose1301 Be sure to download the most recent version of mmd_tools . The export scaling of VMD files is now multiplying, but previously it was dividing because of a weird bug which has now been corrected.

[MarieRose1301]

@Hogarth-MMD I downloaded it today

[Hogarth-MMD]

Be sure that you are using exactly the same bvh and pmx files which you uploaded for me.

[MarieRose1301]

the source animation is not a bvh its tdpack a format used in dead or alive 5 but i will try right now

[Hogarth-MMD]

Import the bvh file as an armature, not to an object, then export it to VMD with a scale of 34.

[MarieRose1301]

oh i get it thanks let me try it

[Hogarth-MMD]

Import the bvh file as an armature in a completely new Blender scene. Don't use the .blend file which you uploaded for me.

[MarieRose1301]

i got it, thank you so much, thought it's still looking very odd, not like it was in blender, or have i just screwed something up again?

[Hogarth-MMD]

In both Blender and MMD, you have a choice between perspective view and orthographic view and the lens angle of the camera affects the result of perspective view in both softwares.

[MarieRose1301]

@Hogarth-MMD i've tried importing the la rotations and put the translations exported by blender on the duplicated bones but it's still not working as expected do you know what could be the problem? i've tried scaling the translations but they don't seem to be correct http://sta.sh/01ki5e5kcsj2 Both are orthographic views

[MarieRose1301]

it seems like movement along Z and Y axis has been swapped while exporting the translations

[powroupi]

Hello, @MarieRose1301 , The size of your model of nyo.blend is not the same as your nyo.pmx, that's why the motion looks different. You may follow the steps below to convert nyo.bvh again:

1. In blender, open a new empty scene.
2. Import nyo.bvh, change BVH import option "Rotation" from "Euler (Native)" to "Quaternion", scale 1.0
3. Export to nyo.vmd, change VMD export option "Scale" from 1.0 to 12.5187674

Now, if you load nyo.pmx and load nyo.vmd in MMD, the result should be the same with in blender when you import nyo.pmx with scale 1.0 and import nyo.vmd with scale 1.0. :smile:

(when I import nyo.pmx to blender with scale 1.0, and import nyo.bvh with scale 1.0, I got the scale 12.5187674 to the bvh model can fit better to pmx model's armature.)

[Hogarth-MMD]

Okay, X 34 is correct scaling for the height of the 2 armatures. But looking at the facial bones exclusively, I am now calculating a scaling of approximately 12.5577 which is similar to the scaling estimate of @powroupi ,

[MarieRose1301]

oh my god thank you both @Hogarth-MMD and @powroupi so much for your help, it finally works, i can't believe it O.O now the only thing left for me to do is get the camera data exported somehow and i'm done with this

[MarieRose1301]

I'm sorry again but I got another question if I may >.> Is camera export supported? Because I've tried exporting it and got nothing

[Hogarth-MMD]

I think it needs to be an MMD camera to export VMD from it(?). There is a button in the properties window which turns the active camera into an MMD camera. The camera is made into a child of an empty object and it is then the empty object which must be animated. Probably @powroupi can give you a better answer.

[MarieRose1301]

Is there a way to keep the camera animation after converting it to mmd cam?

[powroupi]

Is there a way to keep the camera animation after converting it to mmd cam?

Here is a method to apply animation to mmd cam, assume you have a normal camera (animation) object CAM:

1. Bake object's location/rotation to mmd cam:
   1. Create a new camera object MMD_CAM, convert it to mmd cam, the root empty object MMD_CAM_ROOT, go to object properties tab, change MMD_CAM object location to (0, 0, 0)
   2. Create a empty object EMPTY, go to object properties tab, change X rotation to -90 degree, and change parent to CAM object
   3. Add constraint Copy Transforms on MMD_CAM_ROOT, change Target to EMPTY
   4. only select MMD_CAM_ROOT, and bake animation to MMD_CAM_ROOT, enable bake option Only Selected & Visual Keying & Clear Constraints
2. Bake camera lens to mmd cam:
   1. Currently, no easy way. You might key frame Angle manually in MMD Camera Tools panel.
3. Export VMD of MMD_CAM or MMD_CAM_ROOT

[MarieRose1301]

Thank you, but I'm WAY confused because I'm not sure which camera should i copy the frames from... Could you explain using these names so I can understand it better please?

[powroupi]

Well, I'm not sure the structure of your camera animation, is Camera_NYOTENGU_MOT_WIN_120_POR a camera animation? In my opinion, (assume there is no MMD Camera type object in your snapshot), you might try follow my steps above, in step 1 - ii. change the EMPTY's parent to Camera_NYOTENGU_MOT_WIN_120.

And remember to check the scale when you export VMD since the size of model/camera may not be 1:1 to MMD. :)

[MarieRose1301]

I think I understood it However, I get an error on export:

[powroupi]

I guess you forgot to bake animation. Check viewport animation, ensure that MMD_Camera is follwing Empty, so MMD_CAMERA (location/rotation) should be animated the same with Camera_NYOTENGU_MOT_WIN_120_Data.

To bake animation (refer to above step 1 - iv.): a. Only select MMD_Camera, go to Animation toolshelf tab, hit button Bake Action, enable bake option Only Selected & Visual Keying & Clear Constraints b. Export VMD of MMD_Camera

[MarieRose1301]

okay i got it exported though it only exported a 1 frame which isnt anywhere close to what it was supposed to be when i copy transforms from the original camera animation tho i get some weird animation in viewport

[powroupi]

@MarieRose1301 I'm sorry I could not help. :cry: If I have a chance to take a look the blend file, I may be able to get some ideas. :)

[EDIT] Okay, you might try this script doa2mmd_cam.txt. Drag & drop it to blender's text editor, only select/active your camera object Camera_NYOTENGU_MOT_WIN_120 in 3d viewport, then hit Run Script button of text editor. A new mmd camera (with animation) will be created which allow you to export VMD. Hope this helps. :smile:

[MarieRose1301]

Oh my god it works, thank you so much I'm sorry for the trouble though ;-; Although there's something strange maybe I messed something up: https://youtu.be/bvVPnG3F_ck And i'm sorry for replying so late, my internet was down u.u I have the blend file on sta.sh: http://sta.sh/02exx15d5po

[powroupi]

You're welcome, I'm glad that helps. :smile:

Although there's something strange...

It is a limitation that MMD can only keyframe camera angle/fov by integer, we need to refine animation curves manually to make it smooth. :smiley:

And I noticed my script may lose some key frames of camera angle/fov, you might remove last_angle related code (at the end of my script) as follows:

for f in range(scene.frame_start, scene.frame_end):
    scene.frame_set(f)
    angle = atan(cam.data.sensor_height/cam.data.lens/2) * 2
    mmd_cam_root.mmd_camera.angle = angle
    mmd_cam_root.keyframe_insert(data_path='mmd_camera.angle', frame=f)

[EDIT] Here is new script doa2mmd_cam.txt

[MarieRose1301]

it failed to execute when i removed this part also would it work with other camera animations or only the doa ones?

[powroupi]

You can get new script from above edited comment. It will work for similar camera structure as follows:

• Object Target
• Object Parent which has a constraint Track To to Target
  • Object Camera

[EDIT] But it is easy to modify to suit for other structures if you understand blender python scripting. In general, you might only need to modify line#8 to suit for other cases. :smile:

cam_target = cam.parent.constraints[0].target # track to

[MarieRose1301]

I GOT IT NOW Thank you so much, it's working perfectly well The only thing I need to fix is the camera shaking but it's not a big deal Thank you again

[MarieRose1301]

Although I've tried getting CM3D camera to the same setup, I ended up with some very weird results but it did export properly, not sure what i did Here are two blend files do you think you could explain me what should i do? it's the only other game i need camera animations from http://sta.sh/01r3yslf0423 I almost got it working but some areas look pretty off in mmd

[powroupi]

Well, you don't have to change camera structure, simply change line 8 to

cam_target = cam.constraints[0].target # track to

should be able to convert the camera animation of untitled.blend.

I found an issue is that CM3D camera is 4:3, default MMD camera is 16:9, so the camera angle conversion may be wrong. I'm not sure how to fix that at the moment, maybe change MMD screen size, or maybe we need to modify line 52

angle = atan(cam.data.sensor_height/cam.data.lens/2) * 2 # may be incorrect ???

[MarieRose1301]

Thank you so much for your help, I would have never been able to do something i've been wanting to do for a while without you. The camera looks just like it does in game: https://youtu.be/FwZAlmO2R6A Thank you again

[powroupi]

Okay, now mmd_tools can convert camera animation. First select camera target, then select camera object, go to object tab of Properties window, hit convert button of MMD Camera Tools panel, enable Bake Animation (the Scale can control the draw size of MMD camera), then a new MMD Camera rig with converted animation will be created. :smile:

ps. I've improved the performance, so you don't need the above doa2mmd_cam.txt scripts anymore. :)

[ptrthomas]

Related to some of the earlier messages on this thread: Machine-Learning Algorithm Watches Dance Dance Revolution, Then Creates Dances of Its Own.

Yes, one day it will happen. Motion created automatically from the music. In VR. *sigh*

[Hogarth-MMD]

Automatic synchronizing of dance motions to music. Presumably that could be programmed.

[ptrthomas]

It's me again, here to make this thread more of a reference and inspiration to MMD and Blender hackers. You guys are gonna love this if you haven't seen this already. Generation of anime characters using machine learning: http://make.girls.moe/

And here is the "how" - with a lot of pictures and examples (pdf): http://make.girls.moe/technical_report.pdf