search

jianwang-mpi / GlobalEgoMocap

The official implementation of paper: Estimating Egocentric 3D Human Pose in Global Space.
12 stars 4 forks source link

Running on a custom dataset #1

Closed anushl9o5 closed 1 year ago

anushl9o5 commented 1 year ago

Hey! I'm interested in running your paper for a base lining effort.

Can you please clarify what you mean by this? https://github.com/yt4766269/GlobalEgoMocap/blob/main/README.md#prepare-the-data-for-optimization

jianwang-mpi commented 1 year ago

Hello anushl9o5,

The dataset preparation setup is a bit complicated. I just uploaded another directory named "MakeDataForOptimization".

The basic idea was to first get the 2D heatmap and joint depths with the Mo2Cap2 method (or xR-egopose). Then get the egocentric camera pose for each sequence with the OpenVSLAM (or any other camera pose localization method).

The camera poses and body heatmaps/depths are combined into a pkl file, and the human body poses predictions can be further optimized by "optimize_whole_sequence.py".

Do not hesitate to contact me if you have any further questions about my work.

Best,

Jian Wang

anushl9o5 commented 1 year ago

Ok sounds good! I'm setting up your repo to run on your data as a first step. I will be running on custom data shortly after that. I'll bring up any issues I face

anushl9o5 commented 1 year ago

@yt4766269 are there any scripts available to visualize the results from - https://github.com/yt4766269/GlobalEgoMocap#optimize-the-motion-sequences-in-proposed-dataset

anushl9o5 commented 1 year ago

I'm trying to retrain the VAE and I have this error FileNotFoundError: [Errno 2] No such file or directory: '/HPS/ScanNet/static00/SURREAL/smpl_data/seq_names.npy'

jianwang-mpi commented 1 year ago

For the visualization, see utils/skeleton.py, joints_to_mesh function to convert joints to mesh and visualize it with open3d. For training the vae, I am not available today and I will deal with that asap.

jianwang-mpi commented 1 year ago

I uploaded new versions of train_globa.sh and train_local.sh Now they should work.

anushl9o5 commented 1 year ago

Great thanks! What is the path for this /home/jianwang/ScanNet/static00/EgocentricAMASS ?

Follow up: I get the following error when I point it to the directory mentioned here https://github.com/yt4766269/GlobalEgoMocap#train-the-motion-vae Error: ValueError: num_samples should be a positive integer value, but got num_samples=0

jianwang-mpi commented 1 year ago

Can you show the complete call stack?

jianwang-mpi commented 1 year ago

please see recent updates under:

https://github.com/yt4766269/GlobalEgoMocap/blob/main/networks/README.md#motion-vae