SizheAn / mRI

Data repo for mRI: Multi-modal 3D Human Pose Estimation Dataset using mmWave, RGB-D, and Inertial Sensors
Creative Commons Zero v1.0 Universal
52 stars 8 forks source link

frames are different in radar feature and 3D pose label #11

Open tsbiosky opened 1 year ago

tsbiosky commented 1 year ago

I try to use subject{i}_featuremap.npy as training data and 3D pose label as label to training , just like MARS dataset . But I found the frames in each subject{i}_featuremap.npy are different with 3D pose label in subject{i}_all_labels.cpl['naive_gt_kps']

(6384, 14, 14, 5) (6529, 3, 17) (7150, 14, 14, 5) (7205, 3, 17) (6982, 14, 14, 5) (7042, 3, 17) (6893, 14, 14, 5) (6956, 3, 17) (7219, 14, 14, 5) (7243, 3, 17) (6906, 14, 14, 5) (7014, 3, 17) (6743, 14, 14, 5) (8944, 3, 17) (6817, 14, 14, 5) (6876, 3, 17) (6704, 14, 14, 5) (6817, 3, 17) (6911, 14, 14, 5) (6953, 3, 17) (7101, 14, 14, 5) (7334, 3, 17) (6526, 14, 14, 5) (6761, 3, 17) (6876, 14, 14, 5) (7026, 3, 17) (6812, 14, 14, 5) (6981, 3, 17) (6557, 14, 14, 5) (6787, 3, 17) (6722, 14, 14, 5) (6924, 3, 17) (6387, 14, 14, 5) (6602, 3, 17)

I'm not sure how to get aligned dataset with labels

SizheAn commented 1 year ago

Can you check the keys: 'radar_avail_frames'? It specifies the available frames of radar modality. Then you can choose the available ones from the groundtruth.

mhafiz95 commented 1 year ago

I also had the same issue as the person above. The radar_avail_frames have keys [132, 6808] so total available frames are 6940 in this modality. Also, it is not the 'naive_gt_kps' rather it should be the 'gt_avail_frames' as that contains available frames for 3d human joints ground truth. Is this what you're refering to when you're saying to choose the available ones from groundtruth?

SizheAn commented 1 year ago

Sorry for the confusion and I will update a better readme later. Here I'm showing an example:

image

For this subject, there are 6529 frames in total. so 6529 for the shape of gt. The available radar frame starts from index 98 and ends at 6481. As a result, total frames of available radar is 6481-98+1=6384. And this is identical to the shape of radar features. I hope this resolves your problems.

mhafiz95 commented 1 year ago

Thank you so much for the clarifying this issue. This makes more sense. Thanks