search

c-he / NeMF

[NeurIPS 2022] Official implementation of "NeMF: Neural Motion Fields for Kinematic Animation"
MIT License
156 stars 9 forks source link

Questions about the processing of SMPL model and AMASS dataset #1

Open tinatiansjz opened 1 year ago

tinatiansjz commented 1 year ago

Hi Chengan, thank you very much for your amazing work and the open source!

I would like to ask a question about the difference between the provided processed SMPL model and the official one, as mentioned here. What to do to the SMPL model during pre-processing?

c-he commented 1 year ago

Hi Yating, thanks so much for your interest in our work! The original SMPL model is provided in the .pkl format, I just converted it to .npz to align with SMPL-H and SMPL-X. The content is not changed.

tinatiansjz commented 1 year ago

Thank you for clearing this up~ There is one more question about processing the AMASS dataset. The provided AMASS data has a structure shown below:

./data/amass
├── generative
│   ├── test
│   ├── train
│   ├── valid
│   ├── amass.log
│   ├── mean-male-128-30fps.pt
│   └── std-male-128-30fps.pt
├── gmp
│   ├── test
│   ├── train
│   ├── valid
│   ├── amass.log
│   ├── mean-male-128-30fps.pt
│   └── std-male-128-30fps.pt
├── single
│   ├── test
│   ├── train
│   ├── valid
│   ├── amass.log
│   ├── mean-male-128-30fps.pt
│   └── std-male-128-30fps.pt
├── amass_stats.log
└── duration_dist.pdf

What are the differences between generative/, gmp/, and single/? Besides, I tried to adjust the params in configs/amass.yaml but failed to generate mean-male-128-30fps.pt and std-male-128-30fps.pt. Can you give me some suggestions? Thank you for your help.

c-he commented 1 year ago

single/ contains the processed data we used to train the single-motion NeMF, where each AMASS sequence is processed and stored in a separate .pt file. generative/ and gmp contain the data we used to train the generative NeMF and global motion predictor, which are based on different configurations (e.g., unified_orientation is True for the generative data while False for the gmp data).

Generating mean and std files is actually a bit tricky. In addtion to setting normalize to True in the configuration file, you need to remove AMASS splits since mean and std are computed over the entire dataset. To do so, you need to uncomment this code https://github.com/c-he/NeMF/blob/main/src/datasets/amass.py#L544-L546 and then move the generated mean and std files to correct locations.