Pre-trained weights not compatible with the model

[markomih]

Dear authors, thank you for releasing the code.

Could you elaborate on which checkpoint corresponds to the released model? I've tried all four released files, however none of them seems to be compatible with the model. python interhandgen.py --model_path Im2Hands/ref_occ.pt python interhandgen.py --model_path Im2Hands/init_occ.pt python interhandgen.py --model_path Im2Hands/intaghand_baseline.pth python interhandgen.py --model_path Im2Hands/halo_baseline.pt

[jyunlee]

Hello, thank you for your interest in our work! The Google Drive link is now fixed (https://drive.google.com/drive/folders/19Hbfuy7Vg2UVLMNMHbsMKApOS07EZ0lL?usp=drive_link). Please let me know if there are any further issues.

[markomih]

thanks for the quick fix. Tho the demo script still breaks at rendering, here is the log file for:

python interhandgen.py --model_path ./MODELS/ckpt_best.pth INFO:root:Writing log file to exp/default INFO - interhandgen.py - 2024-03-31 15:50:34,477 - Writing log file to exp/default INFO:root:Exp instance id = 4049041 INFO - interhandgen.py - 2024-03-31 15:50:34,477 - Exp instance id = 4049041 c==> Using settings Namespace(seed=19960905, config='default.yml', exp='exp', doc='default', verbose='info', ni=False, actions='*', skip_type='uniform', eta=0.0, sequence=False, model_path='./MODELS/ckpt_best.pth', train=False, log_path='exp/default') ==> Using configures Namespace(data=Namespace(dataset='interhand', dataset_path='/media/STORAGE/InterHand2.6M_processed', num_workers=4), model=Namespace(hid_dim=2056, emd_dim=512, coords_dim=64, num_layer=7, n_head=4, dropout=0.25, ema_rate=0.999, ema=False, var_type='fixedsmall', skips=[1, 3, 5, 6], downsample=1), diffusion=Namespace(beta_schedule='linear', beta_start=0.0001, beta_end=0.01, num_diffusion_timesteps=129), training=Namespace(batch_size=256, lr_gamma=0.9, lr=0.001, decay=60, n_epochs=80, n_iters=5000000, snapshot_freq=5000, validation_freq=2000, vis=False, vis_batch=100), testing=Namespace(test_times=1, test_timesteps=42, test_num_diffusion_timesteps=126, vis=True, vis_batch=250, vis_epoch=4, anti_pen=False), optim=Namespace(decay=20, optimizer='Adam', lr=0.001, lr_gamma=0.9, amsgrad=False, eps=1e-08, grad_clip=4.0), device=device(type='cuda')) Fix shapedirs bug of MANO 0%| | 0/4 [00:00<?, ?it/s]Saving results to exp/default/res... 25%|████████████████████████████████████████▊ | 1/4 [00:01<00:03, 1.10s/it]Saving results to exp/default/res... 50%|█████████████████████████████████████████████████████████████████████████████████▌ | 2/4 [00:01<00:01, 1.34it/s]Saving results to exp/default/res... 75%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▎ | 3/4 [00:02<00:00, 1.55it/s]Saving results to exp/default/res... 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:02<00:00, 1.52it/s] 25%|███████████████████████████████████████▌ | 250/1000 [01:03<03:09, 3.96it/s] ERROR:root:Traceback (most recent call last): File "/media/STORAGE_4TB/projects/InterHandGen/interhandgen.py", line 149, in main runner.test_hyber() File "/media/STORAGE_4TB/projects/InterHandGen/runners/diffhand.py", line 584, in test_hyber imgs, alphas = self.renderer.render( IndexError: list index out of range ERROR - interhandgen.py - 2024-03-31 15:51:42,176 - Traceback (most recent call last): File "/media/STORAGE_4TB/projects/InterHandGen/interhandgen.py", line 149, in main runner.test_hyber() File "/media/STORAGE_4TB/projects/InterHandGen/runners/diffhand.py", line 584, in test_hyber imgs, alphas = self.renderer.render( IndexError: list index out of range

Seems that you are rendering pred_manos instead of total_manos.

[jyunlee]

Thank you for letting me know. 👍 I forgot to change the variable name while refactoring the code. That part is now updated!

[markomih]

Could you please take a look at the evaluation script too :) Running python evaluate.py crushed after 16 iterations. Here's my log:

Start computing evaluation metrics for 10000 samples... Extracting pointnet pred feature... 10%|████████████████▎ | 16/157 [00:06<00:53, 2.64it/s] Traceback (most recent call last): File "/media/STORAGE_4TB/projects/InterHandGen/eval/evaluate.py", line 194, in evaluate(exp, n_samples) File "/media/STORAGE4TB/projects/InterHandGen/eval/evaluate.py", line 81, in evaluate kpts, , actv = model(pred_verts[ip_batch : (i+1)p_batch].cuda(), return_feat=True) File "/home/marko/anaconda3/envs/interhandgen/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl return forward_call(*input, *kwargs) File "/media/STORAGE_4TB/projects/InterHandGen/eval/utils/pointnet2_twohand.py", line 37, in forward l3_xyz, l3_points = self.sa3(l2_xyz, l2_points) File "/home/marko/anaconda3/envs/interhandgen/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl return forward_call(input, **kwargs) File "/media/STORAGE_4TB/projects/InterHandGen/eval/utils/pointnet2_utils.py", line 195, in forward new_xyz, new_points = sample_and_group_all(xyz, points) File "/media/STORAGE_4TB/projects/InterHandGen/eval/utils/pointnet2_utils.py", line 160, in sample_and_group_all new_points = torch.cat([grouped_xyz, points.view(B, 1, N, -1)], dim=-1) RuntimeError: cannot reshape tensor of 0 elements into shape [0, 1, 128, -1] because the unspecified dimension size -1 can be any value and is ambiguous

[jyunlee]

When calling evaluate.py, did you match --sample_num value to the actual number of samples that you previously generated?

P.S. To increase the number of samples to generate (during network inference), you can control vis_epoch and vis_batch values in the config file (https://github.com/jyunlee/InterHandGen?tab=readme-ov-file#network-inference). :)

[markomih]

oh right, I forgot that part. It works fine with 1000 samples for the default config (testing.vis_batch=250, testing.vis_epoch=4). Thanks!