How to individualize SKEL by input shape and pose parameters in Window?

[hwy1992129]

I think the individualization of SKEL is shown in the examples, skel_betas.py, skel_poses.py, but SKELSequence is required. align_to_SMPL.py, working in Window, can individualize the model via AMASS data. How to customize the SKEL without SKELSequence?

[MarilynKeller]

The skel_betas.py and skel_poses.py scripts just aim to show visually the effect of changing the betas (shape) and q (pose) parameters of SKEL. If you want to visualize I suggest you set up a ubuntu virtual machine.

Otherwise, I suggest you play arround with the script quickstart.py as this does not rely on any visualization tool. You can change the values of 'betas' and 'poses' and observe the changes in the output mesh.

betas is a vector of size 10. The values should remain between -2 and 2. The first value of the vector controls the height, the second value the weight.

pose is a vector of size 46, most values are in radiant angles. See here to see which of the 46 parameters controls which body articulation.

[hwy1992129]

@MarilynKeller Thanks for your replied. If I have a SMPL and I know the betas and poses of the SMPL, how to create its corresponding SKEL? betas can be passed directly to SKEL. The pose in SMPL is 72, but I haven't found the code to pass them....

Is the way to do it running the fitting with skel_fitter.fit to calculate 46 from the input SMPL ?

[MarilynKeller]

Yes you are correct, what you need is skel_fitter.py. You can see an example of how to run it in examples/align_to_SMPL.py.

SKEL takes the same betas as SMPL but skel_fitter lets you find the pose parameter q of SKEL that matches the SMPL pose theta.

[MarilynKeller]

I just added some more documentation to clarify examples/align_to_SMPL.py. I suggest you pull the last commit.

[hwy1992129]

Yes you are correct, what you need is skel_fitter.py. You can see an example of how to run it in examples/align_to_SMPL.py.

SKEL takes the same betas as SMPL but skel_fitter lets you find the pose parameter q of SKEL that matches the SMPL pose theta.

align_to_SMPL.py doesn't work with 1 frame, but fitting 2 frames is fine....

Fitting 1 frames
Batch size is larger than the number of frames. Setting batch size to 1
Running batch optimization:   0%|                                                                                                                                                                                                                                 | 0/1 [00:00<?, ?it/s]E:\innovisionllc2024\skel\SKEL\skel\skel_model.py:424: UserWarning: Use of index_put_ on expanded tensors is deprecated. Please clone() the tensor before performing this operation. This also applies to advanced indexing e.g. tensor[indices] = tensor (Triggered internally at  ..\aten\src\ATen\native\TensorAdvancedIndexing.cpp:342.)
  Rsmpl[:, smpl_joint_corresp] = Rskin.clone()[:] # BxNjx3x3 pose params to rotation
Running batch optimization:   0%|                                                                                                                                                                                                                                 | 0/1 [00:02<?, ?it/s]
Traceback (most recent call last):
  File "examples/align_to_SMPL.py", line 39, in <module>
    skel_seq = skel_fitter.fit(smpl_seq['trans'],
  File "E:\skel\SKEL\skel\alignment\aligner.py", line 244, in fit
    optim([trans,poses], poses, betas, trans, verts, self.skel, self.device, rot_only=True, watch_frame=watch_frame)
  File "E:\skel\SKEL\skel\alignment\aligner.py", line 146, in optim
    loss = optimizer.step(closure).item()
  File "C:\Users\hwy19\Miniconda3\envs\skel\lib\site-packages\torch\optim\optimizer.py", line 89, in wrapper
    return func(*args, **kwargs)
  File "C:\Users\hwy19\Miniconda3\envs\skel\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\hwy19\Miniconda3\envs\skel\lib\site-packages\torch\optim\lbfgs.py", line 311, in step
    orig_loss = closure()
  File "C:\Users\hwy19\Miniconda3\envs\skel\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "E:\skel\SKEL\skel\alignment\aligner.py", line 127, in closure
    time_loss = 1e-2 * compute_time_loss(poses)
  File "E:\skel\SKEL\skel\alignment\aligner.py", line 48, in compute_time_loss
    time_loss = torch.linalg.norm(pose_delta, ord=2)
RuntimeError: cannot perform reduction function max on tensor with no elements because the operation does not have an identity