how to generate the npy motion file from bvh?

[visonpon]

Hi @xbpeng ,I have converted my own bvh file to your deepmimic data format success, but when try to convert my bvh to the npy file for ase, the visualized result is unnormal. it seems the rotation value not get right. are there any different for the 4d rotation of each joint between deemimic and ase's data format? or there are other details that i ignored? Hope can receive your responce ,thanks~

[xbpeng]

yes, there are some differences between the deepmimic skeleton and the one in isaacgym. They should be pretty much the same, except in deepmimic, the up vector is the y axis, while in isaacgym the up vector is the z axis, which might explain some of the issues you are seeing. I think the ordering of the quaternions might also be different. I can't recall exactly, but you should check if the w component is the first or last value in a quaternion.

[jonyzhang2023]

Hi all, when I downloaded a fbx file from https://actorcore.reallusion.com/ and tried to convert it to npy file. It succeed but the rotation of every frame is same. How can I fix it?

[erick-alv]

Hello. In poselib there are some motion files of CMU provided as fbx files. Where did you got those files? If you converted the files yourself to fbx format, could you please explain how you did it?

[mJones00]

Hi all, when I downloaded a fbx file from https://actorcore.reallusion.com/ and tried to convert it to npy file. It succeed but the rotation of every frame is same. How can I fix it?

Hello, did you manage to retarget the animation from Reallusion to the skeleton used in the simulation ? As far as I can see, it might need to provide a t-pose for the source animation and a mapping between the joints from the two skeletons. How can you do that ?

[ameliacode]

These are the steps I have went through. Hope this helps anyone stuck in preprocessing.

1. Collect bvh
2. From what I remember I made a python file that converts bvh to fbx, which requires blender python library. (checkout the blender python library) and made a bash file that runs blender python externally for each bvh file. (blender needs to refresh every time for conversion or it may converts the previous accumulated ones :( ) well.. you can achieve this manually by importing bvh and exporting fbx in blender,but for efficient way I recommend using blender python
3. use this: SkeletonMotion.from_fbx and save it to .npy file: motion.to_file(file_path)
4. Retarget npy file for ase with generate_retarget_motion.py
5. I also coded for generating yaml file too.

I made every python files for each steps btw, and yes, I had no problem with orientation either.

[jonyzhang2023]

Hi all, when I downloaded a fbx file from https://actorcore.reallusion.com/ and tried to convert it to npy file. It succeed but the rotation of every frame is same. How can I fix it?

Hello, did you manage to retarget the animation from Reallusion to the skeleton used in the simulation ? As far as I can see, it might need to provide a t-pose for the source animation and a mapping between the joints from the two skeletons. How can you do that ?

I did not import the t-pose into my code so i failed to transfer it. When I just want to visualize the motion data from fbx file, it only has one single frame and this frame looks like t-pose.

ps, I used the motion file from actorcore, which is suggested by this repo. I donot know why it has always only one frame.

[budiu-39]

I did not import the t-pose into my code so i failed to transfer it. When I just want to visualize the motion data from fbx file, it only has one single frame and this frame looks like t-pose.

ps, I used the motion file from actorcore, which is suggested by this repo. I donot know why it has always only one fram

I did same thing as you. And I also got only 1 frame, which might be the T-pose of the character. Maybe there is a code in fbx, which prevent the user directly to read data in fbx of "actorcore.reallusion.com". To protect their data and their model. I had also bought some motion, but they become useless.... QAQ

[3-body]

These are the steps I have went through. Hope this helps anyone stuck in preprocessing.

1. Collect bvh
2. From what I remember I made a python file that converts bvh to fbx, which requires blender python library. (checkout the blender python library) and made a bash file that runs blender python externally for each bvh file. (blender needs to refresh every time for conversion or it may converts the previous accumulated ones :( ) well.. you can achieve this manually by importing bvh and exporting fbx in blender,but for efficient way I recommend using blender python
3. use this: SkeletonMotion.from_fbx and save it to .npy file: motion.to_file(file_path)
4. Retarget npy file for ase with generate_retarget_motion.py
5. I also coded for generating yaml file too.

I made every python files for each steps btw, and yes, I had no problem with orientation either.

Hi, @ameliacode , good work! I just want to code some scripts for these steps, so could u share your wheels?

[jonyzhang2023]

Hi @3-body, @xbpeng

For the step3. running command SkeletonMotion.from_fbx, is it required to installed FBX Autodesk, cause right now I had some trouble with it. The version provided by the website seems too old for python3.8. The latest version as I saw from the website is 3.1

Best,

you can follow the fbx bindings guide, which I just tested in linux. works well.

[jonyzhang2023]

I did not import the t-pose into my code so i failed to transfer it. When I just want to visualize the motion data from fbx file, it only has one single frame and this frame looks like t-pose.

ps, I used the motion file from actorcore, which is suggested by this repo. I donot know why it has always only one fram

I did same thing as you. And I also got only 1 frame, which might be the T-pose of the character. Maybe there is a code in fbx, which prevent the user directly to read data in fbx of "actorcore.reallusion.com". To protect their data and their model. I had also bought some motion, but they become useless.... QAQ

yeah right. I have tried to use other mocap datasets : ) happy to communicate.

[yoyostudy]

@jonyzhang2023

Hello,

Thank you for your reply.

Can I know what python version are you using for the installation? Because the documentation of the fbx binding installation guide page seems old, and only 2.x-3.1 is listed on their website. And I just download the pythonsdk202032 and pythonskd202034, it turns out that one is for python3.7 and the other is for python3.10. But now my virtual environment is using python3.8.

Maybe I should switch to binding as you suggested, could you provide more suggestions.

Thank you so much :=) 🙇‍♂

[jonyzhang2023]

@jonyzhang2023

Hello,

Thank you for your reply.

Can I know what python version are you using for the installation? Because the documentation of the fbx binding installation guide page seems old, and only 2.x-3.1 is listed on their website. And I just download the pythonsdk202032 and pythonskd202034, it turns out that one is for python3.7 and the other is for python3.10. But now my virtual environment is using python3.8.

Maybe I should switch to binding as you suggested, could you provide more suggestions.

Thank you so much :=) 🙇‍♂

Python 3.7 & Python 3.8 work for me. Isaacgym doesnot support python 3.10 so I didnot try that.

[yoyostudy]

@jonyzhang2023

Hello Jony,

Thank you for your reply. Can I know how did you get this visualization of the skeleton, cause I run the test_plotter.py under poselib.poselib.visualization.tests, it returned a blank plot graph for me. And I checked my matplotlib, it seems working.

Thanks :+)

[jonyzhang2023]

@jonyzhang2023

Hello Jony,

Thank you for your reply. Can I know how did you get this visualization of the skeleton, cause I run the test_plotter.py under poselib.poselib.visualization.tests, it returned a blank plot graph for me. And I checked my matplotlib, it seems working.

Thanks :+)

yeah, looks like you do not have fbx sdk on your envs. can you show me more details on your terminal?

[ameliacode]

These are the steps I have went through. Hope this helps anyone stuck in preprocessing.

1. Collect bvh
2. From what I remember I made a python file that converts bvh to fbx, which requires blender python library. (checkout the blender python library) and made a bash file that runs blender python externally for each bvh file. (blender needs to refresh every time for conversion or it may converts the previous accumulated ones :( ) well.. you can achieve this manually by importing bvh and exporting fbx in blender,but for efficient way I recommend using blender python
3. use this: SkeletonMotion.from_fbx and save it to .npy file: motion.to_file(file_path)
4. Retarget npy file for ase with generate_retarget_motion.py
5. I also coded for generating yaml file too.

I made every python files for each steps btw, and yes, I had no problem with orientation either.

Hi, @ameliacode , good work! I just want to code some scripts for these steps, so could u share your wheels?

Oops sorry for late replay. Due to personal issue I can't completely open my code.. However I can give you some hints about this preprocessing though...

For step1, I have imported bpy(blender python) on generating fbx datset. Look for details in official blender python docs for this. Make sure to select all and delete objects first. (there are default objects as cubes, lighting in initial blender project, This may hinder in conversion process). then import bvh files(import_anim.bvh). then select scene object(bvh) then export it as fbx. From what I remember, this example code exists on internet.

I recommend to understand preprocess codes that are in poselib folder in this repository. Good luck

[ghy0626-design]

Hi all, when I downloaded a fbx file from https://actorcore.reallusion.com/ and tried to convert it to npy file. It succeed but the rotation of every frame is same. How can I fix it?

Hello, did you manage to retarget the animation from Reallusion to the skeleton used in the simulation ? As far as I can see, it might need to provide a t-pose for the source animation and a mapping between the joints from the two skeletons. How can you do that ?

I did not import the t-pose into my code so i failed to transfer it. When I just want to visualize the motion data from fbx file, it only has one single frame and this frame looks like t-pose.

ps, I used the motion file from actorcore, which is suggested by this repo. I donot know why it has always only one frame.

Hi, it looks like you have already solved this problem, can I ask how to obtain the T-pose for the 'source_motion'?thanks!