Closed mateuszwyszynski closed 6 months ago
There is a problem to run vposer
with basic SMPL model. It works well with SMPLX. The problem is actually inside the smplx
package itself.
Traceback (most recent call last):
File "/home/mateusz/Programming/viser/./examples/08_smplx_visualizer.py", line 287, in <module>
tyro.cli(main, description=__doc__)
File "/home/mateusz/anaconda3/envs/viser-python-3.9/lib/python3.9/site-packages/tyro/_cli.py", line 187, in cli
output = _cli_impl(
File "/home/mateusz/anaconda3/envs/viser-python-3.9/lib/python3.9/site-packages/tyro/_cli.py", line 454, in _cli_impl
out, consumed_keywords = _calling.call_from_args(
File "/home/mateusz/anaconda3/envs/viser-python-3.9/lib/python3.9/site-packages/tyro/_calling.py", line 241, in call_from_args
return unwrapped_f(*positional_args, **kwargs), consumed_keywords # type: ignore
File "/home/mateusz/Programming/viser/./examples/08_smplx_visualizer.py", line 96, in main
output = model.forward(
File "/home/mateusz/anaconda3/envs/viser-python-3.9/lib/python3.9/site-packages/smplx/body_models.py", line 360, in forward
full_pose = torch.cat([global_orient, body_pose], dim=1)
RuntimeError: Tensors must have same number of dimensions: got 2 and 3
As described in the error, the problem is that the two tensors have different dimensions (2 and 3). I believe that when using SMPLX model the call to the torch.cat
from above is simply different and this is why the code works with the SMPLX model. I have checked the smplx
package code more or less and it seems to be the case. All lines where the full_pose
is constructed are different and incorporate some kind of reshaping (or relate to other model parts like hand_pose
)
As a side note, for some reason body_pose
in the SMPL model has more elements than in the SMPLX model: the shape is torch.Size([1, 23, 3])
instead of torch.Size([1, 21, 3])
. The shape of global_orient
is the same for both models, i.e. torch.Size([1, 3])
. This however most likely is not the cause of the issue - I tried cutting the last two rows (i.e. artificially changing 21 to 23) of body_pose
and still got the error.
I think the best option is to switch to the SMPLX model for the whole repo. It is the newest and the most sophisticated model. At the same time it works smoothly with viser
nerfstudio offers a web-based 3D visualization tool called viser. It works with
smplx
, so I believe we should use it to visualize poses. It will be much more convenient than rendering pictures or videos. One will be able to, for example, look at a pose from different angles or play around with joint parameters.