search

YuliangXiu / ICON

[CVPR'22] ICON: Implicit Clothed humans Obtained from Normals
https://icon.is.tue.mpg.de
Other
1.61k stars 220 forks source link

About Thuman2.1 for icon #253

Open QUANmoxiansheng opened 4 months ago

QUANmoxiansheng commented 4 months ago

run "python -m scripts.render_batch -debug -headless" python -m scripts.render_batch -debug -headless Start Rendering thuman2 with 36 views, 512x512 size. Output dir: ./debug/thuman2_36views Rendering types: ['light', 'normal', 'depth'] 0%| | 0/1 [00:00<?, ?it/s]hands_componentsl:(45, 45) data_struct.hands_componentsr[:num_pca_comps]:(12, 45) Left Hand Pose: tensor([[ 0.0192, -0.0478, -0.6756, 0.0330, -0.0516, -0.6863, 0.0020, 0.0083, -0.3404, -0.1276, -0.0921, -0.8259, -0.0590, 0.0388, -0.5376, -0.0335, 0.0615, -0.3179, -0.2578, -0.1713, -0.3081, -0.3131, -0.1107, -0.4532, -0.0877, -0.0747, -0.3444, -0.1265, -0.1627, -0.5912, -0.0900, -0.0380, -0.5762, -0.0611, 0.0226, -0.3637, 0.0073, 0.0445, 0.0866, 0.0435, 0.1607, 0.0659, 0.0282, 0.2099, -0.0075]]) Left Hand Pose: torch.Size([1, 45]) 0%| | 0/1 [00:02<?, ?it/s] multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/home/b302003/anaconda3/envs/icon/lib/python3.8/multiprocessing/pool.py", line 125, in worker result = (True, func(args, kwds)) File "/home/b302003/zgq/ICON/scripts/render_batch.py", line 74, in render_subject rescale_fitted_body, joints = load_fit_body( File "/home/b302003/zgq/ICON/lib/renderer/mesh.py", line 80, in load_fit_body smpl_out = smpl_model(model_forward_params) File "/home/b302003/anaconda3/envs/icon/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl return forward_call(input, *kwargs) File "/home/b302003/zgq/ICON/lib/smplx/body_models.py", line 1220, in forward left_hand_pose = torch.einsum('bi,ij->bj', [left_hand_pose, self.left_hand_components]) File "/home/b302003/anaconda3/envs/icon/lib/python3.8/site-packages/torch/functional.py", line 373, in einsum return einsum(equation, _operands) File "/home/b302003/anaconda3/envs/icon/lib/python3.8/site-packages/torch/functional.py", line 378, in einsum return _VF.einsum(equation, operands) # type: ignore[attr-defined] RuntimeError: einsum(): subscript i has size 12 for operand 1 which does not broadcast with previously seen size 45 """

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/b302003/anaconda3/envs/icon/lib/python3.8/runpy.py", line 192, in _run_module_as_main return _run_code(code, main_globals, None, File "/home/b302003/anaconda3/envs/icon/lib/python3.8/runpy.py", line 85, in _run_code exec(code, run_globals) File "/home/b302003/zgq/ICON/scripts/renderbatch.py", line 254, in for in tqdm( File "/home/b302003/anaconda3/envs/icon/lib/python3.8/site-packages/tqdm/std.py", line 1180, in iter for obj in iterable: File "/home/b302003/anaconda3/envs/icon/lib/python3.8/multiprocessing/pool.py", line 865, in next raise value RuntimeError: einsum(): subscript i has size 12 for operand 1 which does not broadcast with previously seen size 45 I found that the structure of the hand pose in Thuman2.1 is torch Size([1, 45]) I modified mash.py：num_pca_comps=12→num_pca_comps=45 python -m scripts.render_batch -debug -headless Start Rendering thuman2 with 36 views, 512x512 size. Output dir: ./debug/thuman2_36views Rendering types: ['light', 'normal', 'depth'] 0%| | 0/1 [00:00<?, ?it/s]hands_componentsl:(45, 45) data_struct.hands_componentsr[:num_pca_comps]:(12, 45) Left Hand Pose: tensor([[ 0.0192, -0.0478, -0.6756, 0.0330, -0.0516, -0.6863, 0.0020, 0.0083, -0.3404, -0.1276, -0.0921, -0.8259, -0.0590, 0.0388, -0.5376, -0.0335, 0.0615, -0.3179, -0.2578, -0.1713, -0.3081, -0.3131, -0.1107, -0.4532, -0.0877, -0.0747, -0.3444, -0.1265, -0.1627, -0.5912, -0.0900, -0.0380, -0.5762, -0.0611, 0.0226, -0.3637, 0.0073, 0.0445, 0.0866, 0.0435, 0.1607, 0.0659, 0.0282, 0.2099, -0.0075]]) Left Hand Pose: torch.Size([1, 45]) 0%| | 0/1 [00:02<?, ?it/s] multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/home/b302003/anaconda3/envs/icon/lib/python3.8/multiprocessing/pool.py", line 125, in worker result = (True, func(args, kwds)) File "/home/b302003/zgq/ICON/scripts/render_batch.py", line 74, in render_subject rescale_fitted_body, joints = load_fit_body( File "/home/b302003/zgq/ICON/lib/renderer/mesh.py", line 80, in load_fit_body smpl_out = smpl_model(model_forward_params) File "/home/b302003/anaconda3/envs/icon/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl return forward_call(input, *kwargs) File "/home/b302003/zgq/ICON/lib/smplx/body_models.py", line 1220, in forward left_hand_pose = torch.einsum('bi,ij->bj', [left_hand_pose, self.left_hand_components]) File "/home/b302003/anaconda3/envs/icon/lib/python3.8/site-packages/torch/functional.py", line 373, in einsum return einsum(equation, _operands) File "/home/b302003/anaconda3/envs/icon/lib/python3.8/site-packages/torch/functional.py", line 378, in einsum return _VF.einsum(equation, operands) # type: ignore[attr-defined] RuntimeError: einsum(): subscript i has size 12 for operand 1 which does not broadcast with previously seen size 45 """

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/b302003/anaconda3/envs/icon/lib/python3.8/runpy.py", line 192, in _run_module_as_main return _run_code(code, main_globals, None, File "/home/b302003/anaconda3/envs/icon/lib/python3.8/runpy.py", line 85, in _run_code exec(code, run_globals) File "/home/b302003/zgq/ICON/scripts/renderbatch.py", line 254, in for in tqdm( File "/home/b302003/anaconda3/envs/icon/lib/python3.8/site-packages/tqdm/std.py", line 1180, in iter for obj in iterable: File "/home/b302003/anaconda3/envs/icon/lib/python3.8/multiprocessing/pool.py", line 865, in next raise value RuntimeError: einsum(): subscript i has size 12 for operand 1 which does not broadcast with previously seen size 45 I want to use Thuman2.1, what should I do?

QUANmoxiansheng commented 4 months ago

I modified mash.py：num_pca_comps=12→num_pca_comps=45 python -m scripts.render_batch -debug -headless Start Rendering thuman2 with 36 views, 512x512 size. Output dir: ./debug/thuman2_36views Rendering types: ['light', 'normal', 'depth'] 0%| | 0/1 [00:00<?, ?it/s]hands_componentsl:(45, 45) data_struct.hands_componentsr[:num_pca_comps]:(45, 45) Left Hand Pose: tensor([[ 0.0192, -0.0478, -0.6756, 0.0330, -0.0516, -0.6863, 0.0020, 0.0083, -0.3404, -0.1276, -0.0921, -0.8259, -0.0590, 0.0388, -0.5376, -0.0335, 0.0615, -0.3179, -0.2578, -0.1713, -0.3081, -0.3131, -0.1107, -0.4532, -0.0877, -0.0747, -0.3444, -0.1265, -0.1627, -0.5912, -0.0900, -0.0380, -0.5762, -0.0611, 0.0226, -0.3637, 0.0073, 0.0445, 0.0866, 0.0435, 0.1607, 0.0659, 0.0282, 0.2099, -0.0075]]) Left Hand Pose: torch.Size([1, 45]) left_hand_pose:tensor([[ 0.0393, -0.0674, 0.1712, -0.0793, 0.0294, 0.2559, 0.0135, 0.0411, -0.0908, 0.1081, 0.0022, 0.2151, -0.0212, -0.0391, 0.1139, 0.0069, -0.0152, -0.0979, -0.2720, 0.0942, -0.0404, 0.1984, -0.0339, -0.1092, -0.0774, -0.0360, -0.0559, -0.0490, 0.0692, 0.2110, 0.0424, 0.0240, -0.1428, 0.0380, -0.0761, -0.1258, 0.1199, -0.0399, -0.1013, 0.0099, 0.0114, 0.0472, 0.0509, 0.1342, -0.1714]]) left_hand_pose:torch.Size([1, 45]) 0%| | 0/1 [00:02<?, ?it/s] multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "/home/b302003/anaconda3/envs/icon/lib/python3.8/multiprocessing/pool.py", line 125, in worker result = (True, func(args, kwds)) File "/home/b302003/zgq/ICON/scripts/render_batch.py", line 74, in render_subject rescale_fitted_body, joints = load_fit_body( File "/home/b302003/zgq/ICON/lib/renderer/mesh.py", line 80, in load_fit_body smpl_out = smpl_model(model_forward_params) File "/home/b302003/anaconda3/envs/icon/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl return forward_call(input, **kwargs) File "/home/b302003/zgq/ICON/lib/smplx/body_models.py", line 1263, in forward vertices, joints = lbs( File "/home/b302003/zgq/ICON/lib/smplx/lbs.py", line 207, in lbs pose_offsets = torch.matmul(pose_feature, posedirs).view(batch_size, -1, 3) RuntimeError: expected scalar type Double but found Float """

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/b302003/anaconda3/envs/icon/lib/python3.8/runpy.py", line 192, in _run_module_as_main return _run_code(code, main_globals, None, File "/home/b302003/anaconda3/envs/icon/lib/python3.8/runpy.py", line 85, in _run_code exec(code, run_globals) File "/home/b302003/zgq/ICON/scripts/renderbatch.py", line 254, in for in tqdm( File "/home/b302003/anaconda3/envs/icon/lib/python3.8/site-packages/tqdm/std.py", line 1180, in iter for obj in iterable: File "/home/b302003/anaconda3/envs/icon/lib/python3.8/multiprocessing/pool.py", line 865, in next raise value RuntimeError: expected scalar type Double but found Float

QUANmoxiansheng commented 4 months ago

I want to use Thuman2.1, what should I do?

zjh21 commented 2 months ago

I am facing the same issue. After setting num_pca_comps to 45, I ran into double and float mismatch problems. I just tired to fix all these errors with .double(). Finally, there is a problem with key 'translation'. It seems that the correct key for THuman2.0 should be 'transl'. After that, the rendering process can be run. The results are correct for meshes from THuman2.0.

But I am still faced with incorrect camera problems. For the new meshes from THuman2.1, the rendered images seem to be pointed to the legs. I am still debuging...