Closed ThibaultGROUEIX closed 2 years ago
Hi @ahmedosman ,
I would really appreciate it if you could follow-up on this. Is there a database of STAR parameters with diverse shapes and poses?
Best regards, Thibault
HI @ThibaultGROUEIX ,
Sorry for my late response.
Pose:
Shape:
For the shape, we can simulate diverse body shapes by sampling the betas of STAR (shape parameters) from a Gaussian distribution, because each of the principal components in the shape space is normalized such that +/- 1 beta corresponds to +/-1 standard deviation. The first PC components correspond to the training subject's heights, so +/- 4 beta for the first PC covers more than 99% of the variance of the heights of the subject in the CAESAR and SizeUSA dataset. The second PC components correspond to the subject BMI. Varying +/-3 standard deviation of the first two PC already covers variation in subject height and weight.
In summary, poses from AMASS and sampling the betas from a univariate Gaussian. Feel free to reach out via email if something is not clear or further questions (firstname.lastname@tuebingen.mpg.de) . Can I close the issue now?
Awesome answer, thanks a lot ! Thibault
One last question on this topic if you don't mind.
Why optimize the pose parameters in the conversion script from SMPL to STAR convert_smpl_2_star
if the kinematic tree is the same?
in https://github.com/ahmedosman/STAR/blob/bc6aa8d9be1192ce90d394fdf8898347ec31fe39/star/convertors/losses.py#L67
Why not copy the SMPL pose and just optimize the shape parameters?
Thanks for your answer,
Best,
Thibault
@ThibaultGROUEIX
the SMPL and STAR pose space (kinematic tree and pose deformations) are similar but not exactly identical, there are differences:
As a result of 1 and 2, if you plug pose parameters in both SMPL/STAR the meshes wouldn't be exactly identical, but similar to some extent for most of the poses. So if you drive STAR with SMPL pose parameters from AMASS the motions will be plausible.
In the conversion script, I included the pose/shape parameters if the user is interested in getting a very similar STAR mesh to a given SMPL mesh.
Does this make sense?
best, Ahmed
Thanks Ahmed, I love your detailed answers.
HI @ThibaultGROUEIX ,
Sorry for my late response.
Pose:
* The kinematic tree (rig) of STAR is identical to SMPL, so to get a diverse set of body poses we could use the AMASS dataset (https://amass.is.tue.mpg.de/) to get the pose parameters - directly plugging in the pose parameters into STAR.
Shape:
* For the shape, we can simulate diverse body shapes by sampling the betas of STAR (shape parameters) from a Gaussian distribution, because each of the principal components in the shape space is normalized such that +/- 1 beta corresponds to +/-1 standard deviation. The first PC components correspond to the training subject's heights, so +/- 4 beta for the first PC covers more than 99% of the variance of the heights of the subject in the CAESAR and SizeUSA dataset. The second PC components correspond to the subject BMI. Varying +/-3 standard deviation of the first two PC already covers variation in subject height and weight.
In summary, poses from AMASS and sampling the betas from a univariate Gaussian. Feel free to reach out via email if something is not clear or further questions (firstname.lastname@tuebingen.mpg.de) . Can I close the issue now?
Hey Ahmed and thanks for all of your replies in this thread. I was wondering if you could elaborate more on the pose parameters and how to use AMASS to receive them. I'm trying to generate my own dataset through STAR and wonder how to sample the pose code.
Thanks, Tal
Hi @TalBarami ,
Thanks for your interest in STAR. AMASS uses the SMPL model, so the datasets provide pose and shape parameters for SMPL. If we plug in the pose parameters in SMPL we will get meaningful motions - there should plenty of demos on how to use AMASS parameters to animate SMPL. STAR is a similar model to SMPL but not identical as I mention above, however if you simply use the exact same AMASS pose parameters in STAR you'll get meaningful motions (the meshes won't be identical though).
Thanks, Ahmed
Hi,
Thanks for your awesome work. :) I would like to train an ML task using the STAR model. Do you know what is the biggest / most diverse set of pose and shape parameters for the STAR model available online?
Thanks a lot for your help, Thibault