search

yohanshin / WHAM

MIT License
626 stars 68 forks source link

About AMASS SMPL format #90

Closed uyoung-jeong closed 3 weeks ago

uyoung-jeong commented 3 months ago

First of all, thanks for your great and inspiring work, and also releasing and maintaining the code. I have few questions about SMPL format you used in AMASS for pretraining.

  1. I don't have much experience about handling AMASS dataset, so it might be a dumb question. From the provided document and the code, I am not sure which SMPL model you used for preprocessing. It seems that MotionBERT uses SMPL+H gendered model (link). Is it correct that your method also uses SMPL+H gendered model?

  2. Regardless of the model format, not all the AMASS data provide all the data for the respective model. For example, CNRS and WEIZMANN do not provide SMPL+H data. Then, do you simply ignore those data for pretraining?

  3. If you use gendered model for pretraining, then does it mean that you do not train with gender-neutral model? It seems that female, male, and neutral models have slightly different pose and shape values for the same mesh in AMASS (at least in SMPL-X case). Also, previous works such as 3DCrowdNet and ROMP use gender neutral model for training, since we cannot correctly identify gender on in-the-wild data such as COCO. Is there something I have missed, or do you just pretrain with gendered model while tolerating some degree of error induced by gender?

Torian2002 commented 3 months ago

I'm a newbie in MoCap and want to ask an even dumber question. Should I manually download all 24 of the AMASS datasets used for this project? And what does the optional "render" after AMASS mean?

uyoung-jeong commented 3 weeks ago

I did some experiments for this, and turns out that SMPL+H genered data can be preprocessed with the provided code.