einsum() operands do not broadcast with remapped shapes

[freedenS]

ths for you excellent work! i follow the guide, but met this error

(neuman_env) root@cabdd5912088:/workspace/ml-neuman# CUDA_VISIBLE_DEVICES=7 python render_reposing.py --scene_dir ./data/bike --weights_path ./out/bike_human/checkpoint.pth.tar --motion_name=jumpandroll
reading cameras: 100%|█████████████████████████| 1/1 [00:00<00:00, 27060.03it/s]
reading images meta: 100%|██████████████████| 104/104 [00:00<00:00, 7290.91it/s]
reading point cloud: 100%|████████████| 33742/33742 [00:00<00:00, 206208.52it/s]
Computing near/far for ['bkg']: 100%|█████████| 104/104 [00:01<00:00, 97.03it/s]
using optimized smpl
Traceback (most recent call last):
  File "render_reposing.py", line 199, in <module>
    main(opt)
  File "render_reposing.py", line 135, in main
    smpl_type='optimized'
  File "/workspace/ml-neuman/data_io/neuman_helper.py", line 245, in read_scene
    smpls, world_verts, static_verts, Ts = cls.read_smpls(scene_dir, scene.captures, scale=scale, smpl_type=smpl_type)
  File "/workspace/ml-neuman/data_io/neuman_helper.py", line 303, in read_smpls
    concat_joints=True
  File "/workspace/ml-neuman/models/smpl.py", line 152, in verts_transformations
    return_T=True, concat_joints=concat_joints)
  File "/workspace/ml-neuman/models/smpl.py", line 310, in lbs
    v_shaped = v_template + blend_shapes(betas, shapedirs)
  File "/workspace/ml-neuman/models/smpl.py", line 403, in blend_shapes
    blend_shape = torch.einsum('bl,mkl->bmk', [betas, shape_disps])
  File "/opt/conda/envs/neuman_env/lib/python3.7/site-packages/torch/functional.py", line 406, in einsum
    return einsum(equation, *_operands)
  File "/opt/conda/envs/neuman_env/lib/python3.7/site-packages/torch/functional.py", line 408, in einsum
    return _VF.einsum(equation, operands)  # type: ignore
RuntimeError: einsum() operands do not broadcast with remapped shapes [original->remapped]: [1, 10]->[1, 1, 1, 10] [6890, 3, 300]->[1, 6890, 3, 300]

[jiangwei221]

can you check the tensors are in the correct shape? the shapes at my end are

In [1]: betas.shape
Out[1]: torch.Size([1, 10])

In [2]: shape_disps.shape
Out[2]: torch.Size([6890, 3, 10])

In [3]: torch.einsum('bl,mkl->bmk', [betas, shape_disps]).shape
Out[3]: torch.Size([1, 6890, 3])

[freedenS]

i got the different shape

betas.shape
torch.Size([1, 10])

shape_disps.shape
torch.Size([6890, 3, 300])

i use wrong version of neutral SMPL weights. but there is no netural in 1.0.0

[jiangwei221]

Download neutral SMPL weights(SMPLIFY_CODE_V2.ZIP) from SMPLify, extract basicModel_neutral_lbs_10_207_0_v1.0.0.pkl and rename it to SMPL_NEUTRAL.pkl.

[freedenS]

thank you for your reply!