Concatenation error between pose and sil data using skeletongait++

[nyanmn]

System information (version)

- Pytorch => 2.3.0
- Operating System / Platform => Ubuntu 22.04
- Cuda => 12.0

Steps to reproduce

Heatmap creation CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --nproc_per_node=4 datasets/pretreatment_heatmap.py --pose_data_path=data/ccpg-keypoints --save_root=data/heatmap --dataset_name=CCPG

Creating Symbolic Links for Heatmap and Silhouette Data python datasets/ln_sil_heatmap.py --heatmap_data_path=/workspace/home/user/OpenGait/data/heatmap/CCPG_sigma_8.0_/pkl --silhouette_data_path=/workspace/home/user/OpenGait/data/ccpg-keypoints --output_path=/workspace/home/user/OpenGait/data/linked_outputs

Training SkeletonGait++ CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --nproc_per_node=2 opengait/main.py --cfgs ./configs/skeletongait/skeletongait++_CCPG.yaml --phase train --log_to_file

data_config part for configuration file.

data_cfg:
  dataset_name: CCPG
  dataset_root: /workspace/home/user/OpenGait/data/linked_outputs
  dataset_partition: ./datasets/CCPG/CCPG.json
  num_workers: 1
  data_in_use: [True, True] # heatmap, sil
  remove_no_gallery: false # Remove probe if no gallery for it
  test_dataset_name: CCPG

The error is

File "/workspace/home/user/OpenGait/opengait/modeling/models/skeletongait++.py", line 86, in inputs_pretreament
[rank1]:     cat_data = np.concatenate([pose, sil], axis=1) # [T, 3, H, W]
[rank1]:   File "<__array_function__ internals>", line 180, in concatenate
[rank1]: ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 2, the array at index 0 has size 64 and the array at index 1 has size 17

Those tensors were printed out and the sizes were

print(pose.shape)
print(sil.shape)
(30, 2, 64, 50)
(30, 1, 17, 3)

Issue submission checklist

• [ yes] I checked the problem with documentation, FAQ, issues, and have not found solution.

[jdyjjj]

(30, 1, 17, 3) It appears to be pose data rather than silhouette data. I hope you can check the data again.

[nyanmn]

Thanks for the reply. When symbolic links for heatmap and silhouette data are created, in_sil_heatmap.py code is expecting pkl file for both heatmap and silhouette. But for CCPG dataset, silhouettes are png images, not pkl files. So do I need to convert png to pkl files?

[jdyjjj]

pretreatment.py can help you.

[nyanmn]

Thanks. That is what I need.