FileNotFoundError: [Errno 2] No such file or directory: 'dataset_files/ego_mean_std.pkl'

[MehranRastegarSani]

Hello, First I want to express my appreciation for your fantastic work and contribution. While running your code on Windows 10, I encounter this error.

"FileNotFoundError: [Errno 2] No such file or directory: 'dataset_files/ego_mean_std.pkl'"

Could you please provide me with information on where I can download the "ego_mean_std" file?

[jianwang-mpi]

I uploaded the ego_mean_std file, the problem should have been solved now.

[MehranRastegarSani]

Thanks for providing the missed file. I eventually ran the demo successfully and reached great results. The results that I could see through your visualization script were the skeleton motion of the person. Could you please guide me on how I can show results on the human shape body model like the first two videos you provided on this page? https://people.mpi-inf.mpg.de/~jianwang/projects/egowholemocap/index.html

[jianwang-mpi]

The SMPL human motion is obtained by solving IK on estimated joints. You can refer to vposer and easymocap for the IK solutions. https://github.com/nghorbani/human_body_prior https://github.com/zju3dv/EasyMocap?tab=readme-ov-file

You can also use other ik tools to drive any rigged human model.

[MehranRastegarSani]

Thanks again for your help. Is there a simple way to convert the obtained mo2cap2 key points to the SMPL format? I couldn't find the mo2cap2 format in the standard formats of key points converters provided by mmhuman3d. (https://github.com/open-mmlab/mmhuman3d/blob/main/docs/keypoints_convention.md)

[jianwang-mpi]

The Mo2Cap2 format and other joint formats are under:

https://github.com/jianwang-mpi/egowholemocap/tree/main/mmpose/data/keypoints_mapping

[MehranRastegarSani]

Thanks for the information. Could you please guide me on how to convert from one format to another? I have a script that converts the PKL file to FBX. However, it only works on SMPL key points format. So, I need to convert the mo2cap2 format to SMPL, do you know any script or simple way to do that?

[jianwang-mpi]

There is no simple way to do that. As I said, I use the IK solutions to convert the mo2cap2 joints to smpl parameters. You can follow the https://github.com/nghorbani/human_body_prior to do that.

The basic idea is to modify the loss function in the optimization process in the following way:

1. convert the current SMPL joint optimization results to mo2cap2 format with key points mapping
2. calculate the converted mo2cap2 optimization results with mo2cap2 predictions by the network

such approach can also be seen in the visualization code of motion diffusion models https://github.com/GuyTevet/motion-diffusion-model

[MehranRastegarSani]

Thank you very much for providing such a comprehensive guide.

[MehranRastegarSani]

Thanks again for your guidance. I was finally able to visualize the Egocentric output on a SMPL model. I was wondering what script or software you used to visualize key points like simple points and chains in 3D coordinates. (Please see the attached pictures).

[jianwang-mpi]

I was using blender for rendering.

If you are not familiar with blender, i recommend using pyrender. It can render from python script.